Submit to Sensors Review for Sensors Propose a Special Issue

Journal Browser

► Journal Browser

Visual and Camera Sensors

Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Physical Sensors".

Deadline for manuscript submissions: closed (31 December 2020) | Viewed by 48201

Printed Edition Available!
A printed edition of this Special Issue is available here.

Special issue Visual and Camera Sensors book cover image

Share This Special Issue

Special Issue Editors

Prof. Dr. Kang Ryoung Park

E-Mail Website
Guest Editor

Division of Electronics and Electrical Engineering, Dongguk University, 30, Pildong- ro 1-gil, Jung-gu, Seoul 04620, Republic of Korea
Interests: deep learning; biometrics; image processing
Special Issues, Collections and Topics in MDPI journals

Prof. Dr. Sangyoun Lee

E-Mail Website
Guest Editor

Department of Electrical and Electronic Engineering, Yonsei University, Seoul 03722 Republic of Korea
Interests: human detection and recognition; gesture recognition; face recognition; HEVC
Special Issues, Collections and Topics in MDPI journals

Prof. Dr. Euntai Kim

E-Mail Website1 Website2
Guest Editor

Department of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, Republic of Korea
Interests: pedestrian and vehicle detection; recognition; vision for advanced driver assistance systems (ADAS); robot vision
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Recent developments have led to the widespread use of visual and camera sensors, such as visible light, near-infrared (NIR), and thermal camera sensors, in a variety of applications in video surveillance, biometrics, image compression, computer vision, and image restoration, etc. While existing technology has matured, its performance is still affected by various environmental conditions, and recent approaches have been attempted to use multimodal camera sensors, and fuse deep learning techniques with conventional methods to guarantee higher accuracy. The goal of this Special Issue is to invite high-quality, state-of-the-art research papers that deal with challenging issues in visual and camera sensors. We solicit original papers of unpublished and completed research that are not currently under review by any other conference/magazine/journal. Topics of interest include, but are not limited to, the following:

Image processing, understanding, recognition, compression, reconstruction, and restoration by visible light, NIR, thermal camera, and multimodal camera sensors
Video processing, understanding, recognition, compression, reconstruction, and restoration by various camera sensors
Computer vision by various camera sensors
Biometrics and spoof detection by various camera sensors
Object detection and tracking by various camera sensors
Deep learning by various camera sensors
Approaches that combine deep learning techniques and conventional methods on images by various camera sensors

Prof. Kang Ryoung Park
Prof. Sangyoun Lee
Prof. Euntai Kim
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

Image processing, understanding, recognition, compression, and reconstruction by various camera sensors
Video processing, understanding, recognition, compression, and reconstruction by various camera sensors
Computer vision by various camera sensors
Biometrics by various camera sensors
Deep learning by various camera sensors
Fusion of deep learning and conventional methods by various camera sensors

Published Papers (13 papers)

Download All Papers

Research

18 pages, 7209 KiB

Open AccessArticle

An Optimum Deployment Algorithm of Camera Networks for Open-Pit Mine Slope Monitoring

by Hua Zhang, Pengjie Tao, Xiaoliang Meng, Mengbiao Liu and Xinxia Liu

Sensors 2021, 21(4), 1148; https://doi.org/10.3390/s21041148 - 06 Feb 2021

Cited by 4 | Viewed by 2576

Abstract

With the growth in demand for mineral resources and the increase in open-pit mine safety and production accidents, the intelligent monitoring of open-pit mine safety and production is becoming more and more important. In this paper, we elaborate on the idea of combining the technologies of photogrammetry and camera sensor networks to make full use of open-pit mine video camera resources. We propose the Optimum Camera Deployment algorithm for open-pit mine slope monitoring (OCD4M) to meet the requirements of a high overlap of photogrammetry and full coverage of monitoring. The OCD4M algorithm is validated and analyzed with the simulated conditions of quantity, view angle, and focal length of cameras, at different monitoring distances. To demonstrate the availability and effectiveness of the algorithm, we conducted field tests and developed the mine safety monitoring prototype system which can alert people to slope collapse risks. The simulation’s experimental results show that the algorithm can effectively calculate the optimum quantity of cameras and corresponding coordinates with an accuracy of 30 cm at 500 m (for a given camera). Additionally, the field tests show that the algorithm can effectively guide the deployment of mine cameras and carry out 3D inspection tasks. Full article

(This article belongs to the Special Issue Visual and Camera Sensors)

► Show Figures

Figure 1

12 pages, 3444 KiB

Open AccessArticle

Sky Monitoring System for Flying Object Detection Using 4K Resolution Camera

by Takehiro Kashiyama, Hideaki Sobue and Yoshihide Sekimoto

Sensors 2020, 20(24), 7071; https://doi.org/10.3390/s20247071 - 10 Dec 2020

Cited by 6 | Viewed by 4225

Abstract

The use of drones and other unmanned aerial vehicles has expanded rapidly in recent years. These devices are expected to enter practical use in various fields, such as taking measurements through aerial photography and transporting small and lightweight objects. Simultaneously, concerns over these devices being misused for terrorism or other criminal activities have increased. In response, several sensor systems have been developed to monitor drone flights. In particular, with the recent progress of deep neural network technology, the monitoring of systems using image processing has been proposed. This study developed a monitoring system for flying objects using a 4K camera and a state-of-the-art convolutional neural network model to achieve real-time processing. We installed a monitoring system in a high-rise building in an urban area during this study and evaluated the precision with which it could detect flying objects at different distances under different weather conditions. The results obtained provide important information for determining the accuracy of monitoring systems with image processing in practice. Full article

(This article belongs to the Special Issue Visual and Camera Sensors)

► Show Figures

Figure 1

17 pages, 6259 KiB

Open AccessArticle

Recognition and Grasping of Disorderly Stacked Wood Planks Using a Local Image Patch and Point Pair Feature Method

by Chengyi Xu, Ying Liu, Fenglong Ding and Zilong Zhuang

Sensors 2020, 20(21), 6235; https://doi.org/10.3390/s20216235 - 31 Oct 2020

Cited by 6 | Viewed by 2533

Abstract

Considering the difficult problem of robot recognition and grasping in the scenario of disorderly stacked wooden planks, a recognition and positioning method based on local image features and point pair geometric features is proposed here and we define a local patch point pair feature. First, we used self-developed scanning equipment to collect images of wood boards and a robot to drive a RGB-D camera to collect images of disorderly stacked wooden planks. The image patches cut from these images were input to a convolutional autoencoder to train and obtain a local texture feature descriptor that is robust to changes in perspective. Then, the small image patches around the point pairs of the plank model are extracted, and input into the trained encoder to obtain the feature vector of the image patch, combining the point pair geometric feature information to form a feature description code expressing the characteristics of the plank. After that, the robot drives the RGB-D camera to collect the local image patches of the point pairs in the area to be grasped in the scene of the stacked wooden planks, also obtaining the feature description code of the wooden planks to be grasped. Finally, through the process of point pair feature matching, pose voting and clustering, the pose of the plank to be grasped is determined. The robot grasping experiment here shows that both the recognition rate and grasping success rate of planks are high, reaching 95.3% and 93.8%, respectively. Compared with the traditional point pair feature method (PPF) and other methods, the method present here has obvious advantages and can be applied to stacked wood plank grasping environments. Full article

(This article belongs to the Special Issue Visual and Camera Sensors)

► Show Figures

Figure 1

24 pages, 4838 KiB

Open AccessArticle

Enhanced Image-Based Endoscopic Pathological Site Classification Using an Ensemble of Deep Learning Models

by Dat Tien Nguyen, Min Beom Lee, Tuyen Danh Pham, Ganbayar Batchuluun, Muhammad Arsalan and Kang Ryoung Park

Sensors 2020, 20(21), 5982; https://doi.org/10.3390/s20215982 - 22 Oct 2020

Cited by 19 | Viewed by 2707

Abstract

In vivo diseases such as colorectal cancer and gastric cancer are increasingly occurring in humans. These are two of the most common types of cancer that cause death worldwide. Therefore, the early detection and treatment of these types of cancer are crucial for saving lives. With the advances in technology and image processing techniques, computer-aided diagnosis (CAD) systems have been developed and applied in several medical systems to assist doctors in diagnosing diseases using imaging technology. In this study, we propose a CAD method to preclassify the in vivo endoscopic images into negative (images without evidence of a disease) and positive (images that possibly include pathological sites such as a polyp or suspected regions including complex vascular information) cases. The goal of our study is to assist doctors to focus on the positive frames of endoscopic sequence rather than the negative frames. Consequently, we can help in enhancing the performance and mitigating the efforts of doctors in the diagnosis procedure. Although previous studies were conducted to solve this problem, they were mostly based on a single classification model, thus limiting the classification performance. Thus, we propose the use of multiple classification models based on ensemble learning techniques to enhance the performance of pathological site classification. Through experiments with an open database, we confirmed that the ensemble of multiple deep learning-based models with different network architectures is more efficient for enhancing the performance of pathological site classification using a CAD system as compared to the state-of-the-art methods. Full article

(This article belongs to the Special Issue Visual and Camera Sensors)

► Show Figures

Figure 1

22 pages, 13907 KiB

Open AccessArticle

DoF-Dependent and Equal-Partition Based Lens Distortion Modeling and Calibration Method for Close-Range Photogrammetry

by Xiao Li, Wei Li, Xin’an Yuan, Xiaokang Yin and Xin Ma

Sensors 2020, 20(20), 5934; https://doi.org/10.3390/s20205934 - 20 Oct 2020

Cited by 3 | Viewed by 2740

Abstract

Lens distortion is closely related to the spatial position of depth of field (DoF), especially in close-range photography. The accurate characterization and precise calibration of DoF-dependent distortion are very important to improve the accuracy of close-range vision measurements. In this paper, to meet the need of short-distance and small-focal-length photography, a DoF-dependent and equal-partition based lens distortion modeling and calibration method is proposed. Firstly, considering the direction along the optical axis, a DoF-dependent yet focusing-state-independent distortion model is proposed. By this method, manual adjustment of the focus and zoom rings is avoided, thus eliminating human errors. Secondly, considering the direction perpendicular to the optical axis, to solve the problem of insufficient distortion representations caused by using only one set of coefficients, a 2D-to-3D equal-increment partitioning method for lens distortion is proposed. Accurate characterization of DoF-dependent distortion is thus realized by fusing the distortion partitioning method and the DoF distortion model. Lastly, a calibration control field is designed. After extracting line segments within a partition, the de-coupling calibration of distortion parameters and other camera model parameters is realized. Experiment results shows that the maximum/average projection and angular reconstruction errors of equal-increment partition based DoF distortion model are 0.11 pixels/0.05 pixels and 0.013°/0.011°, respectively. This demonstrates the validity of the lens distortion model and calibration method proposed in this paper. Full article

(This article belongs to the Special Issue Visual and Camera Sensors)

► Show Figures

Figure 1

17 pages, 6405 KiB

Open AccessArticle

Development of a Robust Multi-Scale Featured Local Binary Pattern for Improved Facial Expression Recognition

by Suraiya Yasmin, Refat Khan Pathan, Munmun Biswas, Mayeen Uddin Khandaker and Mohammad Rashed Iqbal Faruque

Sensors 2020, 20(18), 5391; https://doi.org/10.3390/s20185391 - 21 Sep 2020

Cited by 13 | Viewed by 3602

Abstract

Compelling facial expression recognition (FER) processes have been utilized in very successful fields like computer vision, robotics, artificial intelligence, and dynamic texture recognition. However, the FER’s critical problem with traditional local binary pattern (LBP) is the loss of neighboring pixels related to different scales that can affect the texture of facial images. To overcome such limitations, this study describes a new extended LBP method to extract feature vectors from images, detecting each image from facial expressions. The proposed method is based on the bitwise AND operation of two rotational kernels applied on LBP(8,1) and LBP(8,2) and utilizes two accessible datasets. Firstly, the facial parts are detected and the essential components of a face are observed, such as eyes, nose, and lips. The portion of the face is then cropped to reduce the dimensions and an unsharp masking kernel is applied to sharpen the image. The filtered images then go through the feature extraction method and wait for the classification process. Four machine learning classifiers were used to verify the proposed method. This study shows that the proposed multi-scale featured local binary pattern (MSFLBP), together with Support Vector Machine (SVM), outperformed the recent LBP-based state-of-the-art approaches resulting in an accuracy of 99.12% for the Extended Cohn–Kanade (CK+) dataset and 89.08% for the Karolinska Directed Emotional Faces (KDEF) dataset. Full article

(This article belongs to the Special Issue Visual and Camera Sensors)

► Show Figures

Figure 1

37 pages, 11239 KiB

Open AccessArticle

Face and Body-Based Human Recognition by GAN-Based Blur Restoration

by Ja Hyung Koo, Se Woon Cho, Na Rae Baek and Kang Ryoung Park

Sensors 2020, 20(18), 5229; https://doi.org/10.3390/s20185229 - 14 Sep 2020

Cited by 5 | Viewed by 4229

Abstract

The long-distance recognition methods in indoor environments are commonly divided into two categories, namely face recognition and face and body recognition. Cameras are typically installed on ceilings for face recognition. Hence, it is difficult to obtain a front image of an individual. Therefore, in many studies, the face and body information of an individual are combined. However, the distance between the camera and an individual is closer in indoor environments than that in outdoor environments. Therefore, face information is distorted due to motion blur. Several studies have examined deblurring of face images. However, there is a paucity of studies on deblurring of body images. To tackle the blur problem, a recognition method is proposed wherein the blur of body and face images is restored using a generative adversarial network (GAN), and the features of face and body obtained using a deep convolutional neural network (CNN) are used to fuse the matching score. The database developed by us, Dongguk face and body dataset version 2 (DFB-DB2) and ChokePoint dataset, which is an open dataset, were used in this study. The equal error rate (EER) of human recognition in DFB-DB2 and ChokePoint dataset was 7.694% and 5.069%, respectively. The proposed method exhibited better results than the state-of-art methods. Full article

(This article belongs to the Special Issue Visual and Camera Sensors)

► Show Figures

Figure 1

18 pages, 2032 KiB

Open AccessArticle

LdsConv: Learned Depthwise Separable Convolutions by Group Pruning

by Wenxiang Lin, Yan Ding, Hua-Liang Wei, Xinglin Pan and Yutong Zhang

Sensors 2020, 20(15), 4349; https://doi.org/10.3390/s20154349 - 04 Aug 2020

Cited by 1 | Viewed by 4039

Abstract

Standard convolutional filters usually capture unnecessary overlap of features resulting in a waste of computational cost. In this paper, we aim to solve this problem by proposing a novel Learned Depthwise Separable Convolution (LdsConv) operation that is smart but has a strong capacity for learning. It integrates the pruning technique into the design of convolutional filters, formulated as a generic convolutional unit that can be used as a direct replacement of convolutions without any adjustments of the architecture. To show the effectiveness of the proposed method, experiments are carried out using the state-of-the-art convolutional neural networks (CNNs), including ResNet, DenseNet, SE-ResNet and MobileNet, respectively. The results show that by simply replacing the original convolution with LdsConv in these CNNs, it can achieve a significantly improved accuracy while reducing computational cost. For the case of ResNet50, the FLOPs can be reduced by 40.9%, meanwhile the accuracy on the associated ImageNet increases. Full article

(This article belongs to the Special Issue Visual and Camera Sensors)

► Show Figures

Figure 1

33 pages, 8021 KiB

Open AccessArticle

SlimDeblurGAN-Based Motion Deblurring and Marker Detection for Autonomous Drone Landing

by Noi Quang Truong, Young Won Lee, Muhammad Owais, Dat Tien Nguyen, Ganbayar Batchuluun, Tuyen Danh Pham and Kang Ryoung Park

Sensors 2020, 20(14), 3918; https://doi.org/10.3390/s20143918 - 14 Jul 2020

Cited by 17 | Viewed by 3771

Abstract

Deep learning-based marker detection for autonomous drone landing is widely studied, due to its superior detection performance. However, no study was reported to address non-uniform motion-blurred input images, and most of the previous handcrafted and deep learning-based methods failed to operate with these challenging inputs. To solve this problem, we propose a deep learning-based marker detection method for autonomous drone landing, by (1) introducing a two-phase framework of deblurring and object detection, by adopting a slimmed version of deblur generative adversarial network (DeblurGAN) model and a You only look once version 2 (YOLOv2) detector, respectively, and (2) considering the balance between the processing time and accuracy of the system. To this end, we propose a channel-pruning framework for slimming the DeblurGAN model called SlimDeblurGAN, without significant accuracy degradation. The experimental results on the two datasets showed that our proposed method exhibited higher performance and greater robustness than the previous methods, in both deburring and marker detection. Full article

(This article belongs to the Special Issue Visual and Camera Sensors)

► Show Figures

Figure 1

26 pages, 7687 KiB

Open AccessArticle

A Hough-Space-Based Automatic Online Calibration Method for a Side-Rear-View Monitoring System

by Jung Hyun Lee and Dong-Wook Lee

Sensors 2020, 20(12), 3407; https://doi.org/10.3390/s20123407 - 16 Jun 2020

Cited by 3 | Viewed by 3496

Abstract

We propose an automatic camera calibration method for a side-rear-view monitoring system in natural driving environments. The proposed method assumes that the camera is always located near the surface of the vehicle so that it always shoots a part of the vehicle. This method utilizes photographed vehicle information because the captured vehicle always appears stationary in the image, regardless of the surrounding environment. The proposed algorithm detects the vehicle from the image and computes the similarity score between the detected vehicle and the previously stored vehicle model. Conventional online calibration methods use additional equipment or operate only in specific driving environments. On the contrary, the proposed method is advantageous because it can automatically calibrate camera-based monitoring systems in any driving environment without using additional equipment. The calibration range of the automatic calibration method was verified through simulations and evaluated both quantitatively and qualitatively through actual driving experiments. Full article

(This article belongs to the Special Issue Visual and Camera Sensors)

► Show Figures

Figure 1

18 pages, 5318 KiB

Open AccessArticle

SD-VIS: A Fast and Accurate Semi-Direct Monocular Visual-Inertial Simultaneous Localization and Mapping (SLAM)

by Quanpan Liu, Zhengjie Wang and Huan Wang

Sensors 2020, 20(5), 1511; https://doi.org/10.3390/s20051511 - 09 Mar 2020

Cited by 9 | Viewed by 4592

Abstract

In practical applications, how to achieve a perfect balance between high accuracy and computational efficiency can be the main challenge for simultaneous localization and mapping (SLAM). To solve this challenge, we propose SD-VIS, a novel fast and accurate semi-direct visual-inertial SLAM framework, which can estimate camera motion and structure of surrounding sparse scenes. In the initialization procedure, we align the pre-integrated IMU measurements and visual images and calibrate out the metric scale, initial velocity, gravity vector, and gyroscope bias by using multiple view geometry (MVG) theory based on the feature-based method. At the front-end, keyframes are tracked by feature-based method and used for back-end optimization and loop closure detection, while non-keyframes are utilized for fast-tracking by direct method. This strategy makes the system not only have the better real-time performance of direct method, but also have high accuracy and loop closing detection ability based on feature-based method. At the back-end, we propose a sliding window-based tightly-coupled optimization framework, which can get more accurate state estimation by minimizing the visual and IMU measurement errors. In order to limit the computational complexity, we adopt the marginalization strategy to fix the number of keyframes in the sliding window. Experimental evaluation on EuRoC dataset demonstrates the feasibility and superior real-time performance of SD-VIS. Compared with state-of-the-art SLAM systems, we can achieve a better balance between accuracy and speed. Full article

(This article belongs to the Special Issue Visual and Camera Sensors)

► Show Figures

Figure 1

15 pages, 3429 KiB

Open AccessArticle

Camera Calibration with Weighted Direct Linear Transformation and Anisotropic Uncertainties of Image Control Points

by Francesco Barone, Marco Marrazzo and Claudio J. Oton

Sensors 2020, 20(4), 1175; https://doi.org/10.3390/s20041175 - 20 Feb 2020

Cited by 21 | Viewed by 5338

Abstract

Camera calibration is a crucial step for computer vision in many applications. For example, adequate calibration is required in infrared thermography inside gas turbines for blade temperature measurements, for associating each pixel with the corresponding point on the blade 3D model. The blade has to be used as the calibration frame, but it is always only partially visible, and thus, there are few control points. We propose and test a method that exploits the anisotropic uncertainty of the control points and improves the calibration in conditions where the number of control points is limited. Assuming a bivariate Gaussian 2D distribution of the position error of each control point, we set uncertainty areas of control points’ position, which are ellipses (with specific axis lengths and rotations) within which the control points are supposed to be. We use these ellipses to set a weight matrix to be used in a weighted Direct Linear Transformation (wDLT). We present the mathematical formalism for this modified calibration algorithm, and we apply it to calibrate a camera from a picture of a well known object in different situations, comparing its performance to the standard DLT method, showing that the wDLT algorithm provides a more robust and precise solution. We finally discuss the quantitative improvements of the algorithm by varying the modules of random deviations in control points’ positions and with partial occlusion of the object. Full article

(This article belongs to the Special Issue Visual and Camera Sensors)

► Show Figures

Figure 1

17 pages, 4166 KiB

Open AccessArticle

A Self-Assembly Portable Mobile Mapping System for Archeological Reconstruction Based on VSLAM-Photogrammetric Algorithm

by Pedro Ortiz-Coder and Alonso Sánchez-Ríos

Sensors 2019, 19(18), 3952; https://doi.org/10.3390/s19183952 - 12 Sep 2019

Cited by 9 | Viewed by 3253

Abstract

Three Dimensional (3D) models are widely used in clinical applications, geosciences, cultural heritage preservation, and engineering; this, together with new emerging needs such as building information modeling (BIM) develop new data capture techniques and devices with a low cost and reduced learning curve that allow for non-specialized users to employ it. This paper presents a simple, self-assembly device for 3D point clouds data capture with an estimated base price under €2500; furthermore, a workflow for the calculations is described that includes a Visual SLAM-photogrammetric threaded algorithm that has been implemented in C++. Another purpose of this work is to validate the proposed system in BIM working environments. To achieve it, in outdoor tests, several 3D point clouds were obtained and the coordinates of 40 points were obtained by means of this device, with data capture distances ranging between 5 to 20 m. Subsequently, those were compared to the coordinates of the same targets measured by a total station. The Euclidean average distance errors and root mean square errors (RMSEs) ranging between 12–46 mm and 8–33 mm respectively, depending on the data capture distance (5–20 m). Furthermore, the proposed system was compared with a commonly used photogrammetric methodology based on Agisoft Metashape software. The results obtained demonstrate that the proposed system satisfies (in each case) the tolerances of ‘level 1’ (51 mm) and ‘level 2’ (13 mm) for point cloud acquisition in urban design and historic documentation, according to the BIM Guide for 3D Imaging (U.S. General Services). Full article

(This article belongs to the Special Issue Visual and Camera Sensors)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Visual and Camera Sensors

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Published Papers (13 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI