Research

Jump to: Review

11928 KiB

Open AccessArticle

Adapting Local Features for Face Detection in Thermal Image

by Chao Ma, Ngo Thanh Trung, Hideaki Uchiyama, Hajime Nagahara, Atsushi Shimada and Rin-ichiro Taniguchi

Sensors 2017, 17(12), 2741; https://doi.org/10.3390/s17122741 - 27 Nov 2017

Cited by 27 | Viewed by 6895

Abstract

A thermal camera captures the temperature distribution of a scene as a thermal image. In thermal images, facial appearances of different people under different lighting conditions are similar. This is because facial temperature distribution is generally constant and not affected by lighting condition. [...] Read more.

A thermal camera captures the temperature distribution of a scene as a thermal image. In thermal images, facial appearances of different people under different lighting conditions are similar. This is because facial temperature distribution is generally constant and not affected by lighting condition. This similarity in face appearances is advantageous for face detection. To detect faces in thermal images, cascade classifiers with Haar-like features are generally used. However, there are few studies exploring the local features for face detection in thermal images. In this paper, we introduce two approaches relying on local features for face detection in thermal images. First, we create new feature types by extending Multi-Block LBP. We consider a margin around the reference and the generally constant distribution of facial temperature. In this way, we make the features more robust to image noise and more effective for face detection in thermal images. Second, we propose an AdaBoost-based training method to get cascade classifiers with multiple types of local features. These feature types have different advantages. In this way we enhance the description power of local features. We did a hold-out validation experiment and a field experiment. In the hold-out validation experiment, we captured a dataset from 20 participants, comprising 14 males and 6 females. For each participant, we captured 420 images with 10 variations in camera distance, 21 poses, and 2 appearances (participant with/without glasses). We compared the performance of cascade classifiers trained by different sets of the features. The experiment results showed that the proposed approaches effectively improve the performance of face detection in thermal images. In the field experiment, we compared the face detection performance in realistic scenes using thermal and RGB images, and gave discussion based on the results. Full article

(This article belongs to the Special Issue Human Detection, Identification, and Recognition of Gesture and Behavior based on Thermal Camera)

► Show Figures

Figure 1

20611 KiB

Open AccessArticle

Remote Marker-Based Tracking for UAV Landing Using Visible-Light Camera Sensor

by Phong Ha Nguyen, Ki Wan Kim, Young Won Lee and Kang Ryoung Park

Sensors 2017, 17(9), 1987; https://doi.org/10.3390/s17091987 - 30 Aug 2017

Cited by 52 | Viewed by 12014

Abstract

Unmanned aerial vehicles (UAVs), which are commonly known as drones, have proved to be useful not only on the battlefields where manned flight is considered too risky or difficult, but also in everyday life purposes such as surveillance, monitoring, rescue, unmanned cargo, aerial [...] Read more.

Unmanned aerial vehicles (UAVs), which are commonly known as drones, have proved to be useful not only on the battlefields where manned flight is considered too risky or difficult, but also in everyday life purposes such as surveillance, monitoring, rescue, unmanned cargo, aerial video, and photography. More advanced drones make use of global positioning system (GPS) receivers during the navigation and control loop which allows for smart GPS features of drone navigation. However, there are problems if the drones operate in heterogeneous areas with no GPS signal, so it is important to perform research into the development of UAVs with autonomous navigation and landing guidance using computer vision. In this research, we determined how to safely land a drone in the absence of GPS signals using our remote maker-based tracking algorithm based on the visible light camera sensor. The proposed method uses a unique marker designed as a tracking target during landing procedures. Experimental results show that our method significantly outperforms state-of-the-art object trackers in terms of both accuracy and processing time, and we perform test on an embedded system in various environments. Full article

(This article belongs to the Special Issue Human Detection, Identification, and Recognition of Gesture and Behavior based on Thermal Camera)

► Show Figures

Figure 1

16326 KiB

Open AccessArticle

Efficient Pedestrian Detection at Nighttime Using a Thermal Camera

by Jeonghyun Baek, Sungjun Hong, Jisu Kim and Euntai Kim

Sensors 2017, 17(8), 1850; https://doi.org/10.3390/s17081850 - 10 Aug 2017

Cited by 58 | Viewed by 6433

Abstract

Most of the commercial nighttime pedestrian detection (PD) methods reported previously utilized the histogram of oriented gradient (HOG) or the local binary pattern (LBP) as the feature and the support vector machine (SVM) as the classifier using thermal camera images. In this paper, [...] Read more.

Most of the commercial nighttime pedestrian detection (PD) methods reported previously utilized the histogram of oriented gradient (HOG) or the local binary pattern (LBP) as the feature and the support vector machine (SVM) as the classifier using thermal camera images. In this paper, we propose a new feature called the thermal-position-intensity-histogram of oriented gradient (TPIHOG or T

π

HOG) and developed a new combination of the T

π

HOG and the additive kernel SVM (AKSVM) for efficient nighttime pedestrian detection. The proposed T

π

HOG includes detailed information on gradient location; therefore, it has more distinctive power than the HOG. The AKSVM performs better than the linear SVM in terms of detection performance, while it is much faster than other kernel SVMs. The combined T

π

HOG-AKSVM showed effective nighttime PD performance with fast computational time. The proposed method was experimentally tested with the KAIST pedestrian dataset and showed better performance compared with other conventional methods. Full article

(This article belongs to the Special Issue Human Detection, Identification, and Recognition of Gesture and Behavior based on Thermal Camera)

► Show Figures

Figure 1

6820 KiB

Open AccessArticle

Thermal Infrared Pedestrian Image Segmentation Using Level Set Method

by Yulong Qiao, Ziwei Wei and Yan Zhao

Sensors 2017, 17(8), 1811; https://doi.org/10.3390/s17081811 - 06 Aug 2017

Cited by 26 | Viewed by 5317

Abstract

The edge-based active contour model has been one of the most influential models in image segmentation, in which the level set method is usually used to minimize the active contour energy function and then find the desired contour. However, for infrared thermal pedestrian [...] Read more.

The edge-based active contour model has been one of the most influential models in image segmentation, in which the level set method is usually used to minimize the active contour energy function and then find the desired contour. However, for infrared thermal pedestrian images, the traditional level set-based method that utilizes the gradient information as edge indicator function fails to provide the satisfactory boundary of the target. That is due to the poorly defined boundaries and the intensity inhomogeneity. Therefore, we propose a novel level set-based thermal infrared image segmentation method that is able to deal with the above problems. Specifically, we firstly explore the one-bit transform convolution kernel and define a soft mark, from which the target boundary is enhanced. Then we propose a weight function to adaptively adjust the intensity of the infrared image so as to reduce the intensity inhomogeneity. In the level set formulation, those processes can adaptively adjust the edge indicator function, from which the evolving curve will stop at the target boundary. We conduct the experiments on benchmark infrared pedestrian images and compare our introduced method with the state-of-the-art approaches to demonstrate the excellent performance of the proposed method. Full article

(This article belongs to the Special Issue Human Detection, Identification, and Recognition of Gesture and Behavior based on Thermal Camera)

► Show Figures

Figure 1

6431 KiB

Open AccessArticle

Person Recognition System Based on a Combination of Body Images from Visible Light and Thermal Cameras

by Dat Tien Nguyen, Hyung Gil Hong, Ki Wan Kim and Kang Ryoung Park

Sensors 2017, 17(3), 605; https://doi.org/10.3390/s17030605 - 16 Mar 2017

Cited by 390 | Viewed by 11375

Abstract

The human body contains identity information that can be used for the person recognition (verification/recognition) problem. In this paper, we propose a person recognition method using the information extracted from body images. Our research is novel in the following three ways compared to [...] Read more.

The human body contains identity information that can be used for the person recognition (verification/recognition) problem. In this paper, we propose a person recognition method using the information extracted from body images. Our research is novel in the following three ways compared to previous studies. First, we use the images of human body for recognizing individuals. To overcome the limitations of previous studies on body-based person recognition that use only visible light images for recognition, we use human body images captured by two different kinds of camera, including a visible light camera and a thermal camera. The use of two different kinds of body image helps us to reduce the effects of noise, background, and variation in the appearance of a human body. Second, we apply a state-of-the art method, called convolutional neural network (CNN) among various available methods, for image features extraction in order to overcome the limitations of traditional hand-designed image feature extraction methods. Finally, with the extracted image features from body images, the recognition task is performed by measuring the distance between the input and enrolled samples. The experimental results show that the proposed method is efficient for enhancing recognition accuracy compared to systems that use only visible light or thermal images of the human body. Full article

(This article belongs to the Special Issue Human Detection, Identification, and Recognition of Gesture and Behavior based on Thermal Camera)

► Show Figures

Figure 1

2218 KiB

Open AccessArticle

Supportive Noninvasive Tool for the Diagnosis of Breast Cancer Using a Thermographic Camera as Sensor

by Marco Antonio Garduño-Ramón, Sofia Giovanna Vega-Mancilla, Luis Alberto Morales-Henández and Roque Alfredo Osornio-Rios

Sensors 2017, 17(3), 497; https://doi.org/10.3390/s17030497 - 03 Mar 2017

Cited by 45 | Viewed by 8075

Abstract

Breast cancer is the leading disease in incidence and mortality among women in developing countries. The opportune diagnosis of this disease strengthens the survival index. Mammography application is limited by age and periodicity. Temperature is a physical magnitude that can be measured by [...] Read more.

Breast cancer is the leading disease in incidence and mortality among women in developing countries. The opportune diagnosis of this disease strengthens the survival index. Mammography application is limited by age and periodicity. Temperature is a physical magnitude that can be measured by using multiple sensing techniques. IR (infrared) thermography using commercial cameras is gaining relevance in industrial and medical applications because it is a non-invasive and non-intrusive technology. Asymmetrical temperature in certain human body zones is associated with cancer. In this paper, an IR thermographic sensor is applied for breast cancer detection. This work includes an automatic breast segmentation methodology, to spot the hottest regions in thermograms using the morphological watershed operator to help the experts locate the tumor. A protocol for thermogram acquisition considering the required time to achieve a thermal stabilization is also proposed. Breast thermograms are evaluated as thermal matrices, instead of gray scale or false color images, increasing the certainty of the provided diagnosis. The proposed tool was validated using the Database for Mastology Research and tested in a voluntary group of 454 women of different ages and cancer stages with good results, leading to the possibility of being used as a supportive tool to detect breast cancer and angiogenesis cases. Full article

(This article belongs to the Special Issue Human Detection, Identification, and Recognition of Gesture and Behavior based on Thermal Camera)

► Show Figures

Figure 1

12991 KiB

Open AccessArticle

Tracking and Classification of In-Air Hand Gesture Based on Thermal Guided Joint Filter

by Seongwan Kim, Yuseok Ban and Sangyoun Lee

Sensors 2017, 17(1), 166; https://doi.org/10.3390/s17010166 - 17 Jan 2017

Cited by 15 | Viewed by 5894

Abstract

The research on hand gestures has attracted many image processing-related studies, as it intuitively conveys the intention of a human as it pertains to motional meaning. Various sensors have been used to exploit the advantages of different modalities for the extraction of important [...] Read more.

The research on hand gestures has attracted many image processing-related studies, as it intuitively conveys the intention of a human as it pertains to motional meaning. Various sensors have been used to exploit the advantages of different modalities for the extraction of important information conveyed by the hand gesture of a user. Although many works have focused on learning the benefits of thermal information from thermal cameras, most have focused on face recognition or human body detection, rather than hand gesture recognition. Additionally, the majority of the works that take advantage of multiple modalities (e.g., the combination of a thermal sensor and a visual sensor), usually adopting simple fusion approaches between the two modalities. As both thermal sensors and visual sensors have their own shortcomings and strengths, we propose a novel joint filter-based hand gesture recognition method to simultaneously exploit the strengths and compensate the shortcomings of each. Our study is motivated by the investigation of the mutual supplementation between thermal and visual information in low feature level for the consistent representation of a hand in the presence of varying lighting conditions. Accordingly, our proposed method leverages the thermal sensor’s stability against luminance and the visual sensors textural detail, while complementing the low resolution and halo effect of thermal sensors and the weakness against illumination of visual sensors. A conventional region tracking method and a deep convolutional neural network have been leveraged to track the trajectory of a hand gesture and to recognize the hand gesture, respectively. Our experimental results show stability in recognizing a hand gesture against varying lighting conditions based on the contribution of the joint kernels of spatial adjacency and thermal range similarity. Full article

(This article belongs to the Special Issue Human Detection, Identification, and Recognition of Gesture and Behavior based on Thermal Camera)

► Show Figures

Figure 1

Review

Jump to: Research

11798 KiB

Open AccessReview

A Survey on Banknote Recognition Methods by Various Sensors

by Ji Woo Lee, Hyung Gil Hong, Ki Wan Kim and Kang Ryoung Park

Sensors 2017, 17(2), 313; https://doi.org/10.3390/s17020313 - 08 Feb 2017

Cited by 50 | Viewed by 15428

Abstract

Despite a decrease in the use of currency due to the recent growth in the use of electronic financial transactions, real money transactions remain very important in the global market. While performing transactions with real money, touching and counting notes by hand, is [...] Read more.

Despite a decrease in the use of currency due to the recent growth in the use of electronic financial transactions, real money transactions remain very important in the global market. While performing transactions with real money, touching and counting notes by hand, is still a common practice in daily life, various types of automated machines, such as ATMs and banknote counters, are essential for large-scale and safe transactions. This paper presents studies that have been conducted in four major areas of research (banknote recognition, counterfeit banknote detection, serial number recognition, and fitness classification) in the accurate banknote recognition field by various sensors in such automated machines, and describes the advantages and drawbacks of the methods presented in those studies. While to a limited extent some surveys have been presented in previous studies in the areas of banknote recognition or counterfeit banknote recognition, this paper is the first of its kind to review all four areas. Techniques used in each of the four areas recognize banknote information (denomination, serial number, authenticity, and physical condition) based on image or sensor data, and are actually applied to banknote processing machines across the world. This study also describes the technological challenges faced by such banknote recognition techniques and presents future directions of research to overcome them. Full article

(This article belongs to the Special Issue Human Detection, Identification, and Recognition of Gesture and Behavior based on Thermal Camera)

► Show Figures