Detection of “Legbreaker” Antipersonnel Landmines by Analysis of Aerial Thermographic Images of the Soil ^†

Abstract

An automatic detection methodology for “legbreaker” Antipersonnel Landmines (APL) was developed based on digital image processing techniques and pattern recognition, applied to thermal images acquired by means of an Unmanned Aerial Vehicle (UAV) equipped with a thermal camera. The images were acquired from the inspection of a natural terrain with sparse vegetation and under uncontrolled conditions, in which prototypes of “legbreaker” APL were buried at different depths. Remarkable results were obtained using a Multilayer Perceptron (MLP) classifier, reaching a 97.1% success rate in detecting areas with the presence of these artifacts.

1. Introduction

“Legbreaker” APL are explosive devices designed to undermine the integrity of a person, causing muscle injuries, limb amputations or even death. Its use is very common by various terrorist groups, and it is estimated that there are more than 110 million mines spread over more than 64 countries, including Colombia, which affect 26,000 people each year [1].

According to the literature, the most effective and common methods for the detection of APL require close physical contact with the area to be inspected, which puts the lives of those involved during this work at risk. Although there are methods that allow remote evaluation such as metal detectors [2,3] or ground penetration radars [4,5], the success of their detection depends on the type and depth of the APL. More recently, the use of UAV carrying multispectral devices has provided promising remote solutions [6,7]. On the other hand, the use of Infrared Thermography (IT) is under development as a viable method for APL detection [8,9,10,11], either in a static or a UAV configuration, based on the thermal difference between the mine and the surrounding soil. However, most of the works that have undertaken this objective were developed under controlled experimental conditions, so these results may differ significantly from what may occur in a real-life situation. With the aim of contributing to this research field, an artificial vision methodology is proposed consisting of the acquisition of aerial thermal images of the terrain using a UAV, under uncontrolled experimental conditions, and their treatment with digital image processing and pattern recognition techniques.

2. Materials and Methods

The APL automatic detection methodology was conceived as a collection of several sequential processing stages, which will be explained next.

2.1. Experiment Design

According to common structures of APL found in Colombia [12], 8 antipersonnel mines were built with an external 10.2 cm height and 8.7 cm diameter PVC cylinder and filled with approximately 20 g of nails and 400 g of anthracite coal; finally, a 5 mL syringe as the bomb fuse was introduced in this cylinder (Figure 1a). Anthracite coal and TNT have similar characteristics, which allows anthracite to be used as an explosive replacement. The APL built were buried in a square soil area of approximately 10 m beside the university campus at 1, 5 and 10 cm depth, according to the distribution shown in Figure 1b. Additionally, some APL were put superficially as a reference for the processing system.

2.2. Image Acquisition

The aerial thermal images, with a size of 336 × 256 pixels, were acquired using a DJI Zenmuse XT camera mounted on a DJI Matrice 100 drone. This camera offers a 50 mK thermal sensitivity in the 7.5–13.5 μm spectral band. The experiment consisted of placing the UAV perpendicular to each zone of the grid of Figure 1b at an altitude of 10 m to capture images in JPG, TIFF and R-JPG formats; then, the UAV descended 1 m, repeating the acquisition procedure, and so on until it reached an altitude of 1 m from the ground. Once the nine inspection areas were scanned, the drone was placed 1 m over the first zone to take 10 images with a difference of 1 s between them. This last procedure was repeated through all the zones of the grid to have more images taken at 1 m from the ground, where the thermal contrast between mines and surrounding soil could be better captured. Finally, the acquired dataset for only one experiment consisted of 900 aerial thermal images of the inspected terrain, in each of the previous file formats, resulting from 5 inspection overflights over the 9 areas of the grid, at the 10 predefined altitudes and at 1 m.

According to the conclusions in [11], the most suitable conditions for the inspection correspond to a sunny day, without cloud cover or rain. Therefore, most of the experiments were carried out after several days of dry weather; a few experiments were carried out after some days of rain to add variability in the soil moisture condition, a parameter that, in excess, negatively affects the thermographic inspection. Additionally, all inspections were made between 5:00 p.m. and 6:00 p.m., when the best thermal contrast between APL and soil is presented, increasing the detectability of the buried artifacts [11].

Table 1. Average and standard deviation of environmental variables throughout experiments.

Ambient Temperature	Solar Radiation	Relative Humidity	Wind Speed
25.9 °C	119.9 W/m2	69.8%	1.9 m/s
±1.2 °C	±5.6 W/m2	±7.7%	±0.7 m/s

shows the average environmental conditions throughout the experiments.

2.3. Image Preprocessing–Dataset Organization–Extraction and Selection of Features

The acquired aerial thermal images were filtered by means of a median filtering with a 3 × 3 pixels kernel to reduce the intrinsic noise of the camera while still preserving edges that help in detecting regions likely to contain buried mines.

To train the APL detection system, the best contrasted 136 images from a set of 198 images acquired at an altitude of 1 m were used. From each of these images, 8 Regions of Interest (ROI) with dimensions of 16 × 16 pixels were segmented to extract features: 4 ROI corresponding to buried APL region and the other 4 ROI from clean areas, obtaining a total of 1088 ROI. For each ROI, a set of 22 characteristics was extracted and normalized in the range from 0 to 1: the 4 first statistical moments around the mean of intensities, the maximum and minimum intensities, and energy, contrast, correlation and homogeneity of the co-occurrence matrices at 0 °, 45 °, 90 ° and 135 °. After analyzing these normalized characteristics with the Fisher’s Discriminant Ratio (FDR) criterion and the scalar selection technique, the mean, the minimum and maximum values, and the energy of the co-occurrence matrix at 90 ° were categorized as the most discriminating.

2.4. Classification with Machine Learning

As a classification method, an Artificial Neural Network type MLP with 15 neurons in the hidden layer was selected and trained with the values of the set of discriminant characteristics extracted from the training images (Section 2.3) and normalized in the range of [0, 1]. The Levenberg–Marquardt algorithm was chosen as the training method. The overall percentage of training achieved with this configuration was 99.4%.

However, during the classification stage with the previously trained MLP, a windowing technique was implemented with the aim of making a total scan over each image to be classified. For this purpose, 136 images were randomly selected from the original set of 198 images; but this time, for each image, 4941 ROI of 16 × 16 pixels were extracted, sweeping the image with displacements of 4 pixels horizontally and vertically. Then, the trained classifier classified every ROI as an empty or suspicious region in order to combine these results into a binary image from which a potential area of any existing, buried APL was segmented; finally, the contour of this area was superposed to the original thermal image.

3. Results and Discussion

The success rate of the MLP classifier in detecting APL reached 97.1% when classifying a validation set of 136 thermal images acquired at 1 m from the ground. Figure 2 shows some of these detection results in the buried APL zones (1, 3, 6, 7, 8 and 9 on the grid of Figure 1b), with two sets of images taken on different days and therefore different environmental conditions. Additionally, the classifier presents a good generalization, represented by a success detection rate of 88.8% in a set of 162 images acquired at an altitude greater than 1 m, despite having been trained with image samples acquired at 1 m. Figure 3 shows the results obtained for zone 8 of the inspection grid, when the altitude was varied from 10 to 1 m. Figure 4 shows the evolution of the detection rate with altitude.

It is important to mention that the raw results of the classification present some false positives that were eliminated through a post-processing stage (Figure 2 and Figure 3), based on the premise that any segmented region whose area is less than that of the approximate sectional area of a mine will not be considered as a mined area. For that, it is necessary to obtain the distance/pixel relationship (Lpq) in the q direction (horizontal or vertical) as a function of the dimension in q of each image in pixels (dq), the angle of view in q of the camera (θq) and the altitude from the ground (h), according to Equation (1).

Lpq = [2h tan(θq/2)]/dq

(1)

On the other hand, although the humidity accumulated in the soil had a negative effect on the thermal contrast, the MLP classifier offered an acceptable performance in APL detection under this condition from images taken at low altitudes (less than 3 m). However, additional testing is required to quantify this robustness of the MLP.

4. Conclusions

An automatic MAP detection methodology based on an MLP was proposed that works with a set of statistical and texture characteristics with the highest discriminant power according to the FDR criterion, extracted with a windowing technique from aerial thermal images acquired by means of a UAV at different altitudes between 1 and 10 m. The proposed methodology has a success rate of 97.1% in detecting APL from images acquired at 1 m from ground and a success rate of 88.8% in detecting APL from images taken at higher altitudes, up to 10 m. This last result demonstrates the robustness of the MLP in this classification problem, knowing that it was trained only with images acquired at 1 m above the ground.

All thermographic inspection tests were carried out between 5:00 and 6:00 pm, which turns out to be the best time slot to maximize the thermal contrast between mines and surrounding soil for the geographical location of the terrain. The excess moisture stored in the soil is a condition that, while hindering thermographic inspection, can be compensated to some degree by learning machines.