Sensors

Research

15 pages, 4819 KiB

Open AccessArticle

The Duality of Ray-Based and Pinhole-Camera Modeling and 3D Measurement Improvements Using the Ray-Based Model

by Christian Bräuer-Burchardt, Roland Ramm, Peter Kühmstedt and Gunther Notni

Sensors 2022, 22(19), 7540; https://doi.org/10.3390/s22197540 - 5 Oct 2022

Cited by 3 | Viewed by 1463

Geometrical camera modeling is the precondition for 3D-reconstruction tasks using photogrammetric sensor systems. The purpose of this study is to describe an approach for possible accuracy improvements by using the ray-based-camera model. The relations between the common pinhole and the generally valid ray-based-camera [...] Read more.

Geometrical camera modeling is the precondition for 3D-reconstruction tasks using photogrammetric sensor systems. The purpose of this study is to describe an approach for possible accuracy improvements by using the ray-based-camera model. The relations between the common pinhole and the generally valid ray-based-camera model are shown. A new approach to the implementation and calibration of the ray-based-camera model is introduced. Using a simple laboratory setup consisting of two cameras and a projector, experimental measurements were performed. The experiments and results showed the possibility of easily transforming the common pinhole model into a ray-based model and of performing calibration using the ray-based model. These initial results show the model’s potential for considerable accuracy improvements, especially for sensor systems using wide-angle lenses or with deep 3D measurements. This study presents several approaches for further improvements to and the practical usage of high-precision optical 3D measurements. Full article

(This article belongs to the Special Issue Camera Calibration and 3D Reconstruction)

► Show Figures

Figure 1

22 pages, 1245 KiB

Open AccessArticle

Computer-Vision-Based Vibration Tracking Using a Digital Camera: A Sparse-Optical-Flow-Based Target Tracking Method

by Guang-Yu Nie, Saran Srikanth Bodda, Harleen Kaur Sandhu, Kevin Han and Abhinav Gupta

Sensors 2022, 22(18), 6869; https://doi.org/10.3390/s22186869 - 11 Sep 2022

Cited by 8 | Viewed by 3253

Abstract

Computer-vision-based target tracking is a technology applied to a wide range of research areas, including structural vibration monitoring. However, current target tracking methods suffer from noise in digital image processing. In this paper, a new target tracking method based on the sparse optical [...] Read more.

Computer-vision-based target tracking is a technology applied to a wide range of research areas, including structural vibration monitoring. However, current target tracking methods suffer from noise in digital image processing. In this paper, a new target tracking method based on the sparse optical flow technique is introduced for improving the accuracy in tracking the target, especially when the target has a large displacement. The proposed method utilizes the Oriented FAST and Rotated BRIEF (ORB) technique which is based on FAST (Features from Accelerated Segment Test), a feature detector, and BRIEF (Binary Robust Independent Elementary Features), a binary descriptor. ORB maintains a variety of keypoints and combines the multi-level strategy with an optical flow algorithm to search the keypoints with a large motion vector for tracking. Then, an outlier removal method based on Hamming distance and interquartile range (IQR) score is introduced to minimize the error. The proposed target tracking method is verified through a lab experiment—a three-story shear building structure subjected to various harmonic excitations. It is compared with existing sparse-optical-flow-based target tracking methods and target tracking methods based on three other types of techniques, i.e., feature matching, dense optical flow, and template matching. The results show that the performance of target tracking is greatly improved through the use of a multi-level strategy and the proposed outlier removal method. The proposed sparse-optical-flow-based target tracking method achieves the best accuracy compared to other existing target tracking methods. Full article

(This article belongs to the Special Issue Camera Calibration and 3D Reconstruction)

► Show Figures

Figure 1

18 pages, 6540 KiB

Open AccessArticle

Phase Target-Based Calibration of Projector Radial Chromatic Aberration for Color Fringe 3D Measurement Systems

by Yuzhuo Zhang, Yaqin Sun, Nan Gao, Zhaozong Meng and Zonghua Zhang

Sensors 2022, 22(18), 6845; https://doi.org/10.3390/s22186845 - 9 Sep 2022

Viewed by 1773

Abstract

The camera and projector are indispensable hardware parts of a color fringe projection 3D measurement system. Chromatic aberration between different color channels of the projector and camera has an impact on the measurement accuracy of the color fringe projection 3D profile measurement. There [...] Read more.

The camera and projector are indispensable hardware parts of a color fringe projection 3D measurement system. Chromatic aberration between different color channels of the projector and camera has an impact on the measurement accuracy of the color fringe projection 3D profile measurement. There are many studies on camera calibration, but the chromatic aberration of the projector remains a question deserving of further investigation. In view of the complex system architecture and theoretical derivation of the traditional projector radial chromatic aberration method, a phase target based on projector radial chromatic aberration measurement and the correction method are proposed in this paper. This method uses a liquid crystal display with a holographic projection film as the phase target. The liquid crystal display sequentially displays red, green, and blue horizontal and vertical sinusoidal fringe images. The projector projects red, green, and blue horizontal and vertical sinusoidal fringe images to the phase target in turn, and calculates the absolute phases of the display fringes and reflection fringes, respectively. Taking the green channel as the reference channel, a phase coordinate system is established based on the phases of the vertical and horizontal directions displayed on the display screen, using the phase of the reflection fringes on the display screen as the ideal phase value of the phase point. Then, the phase coordinate system of the red and blue channels is transferred to the green phase coordinate system to calculate the chromatic aberration of the red-green channels and the blue-green channels, and pre-compensation is conducted. Experimental results prove that this method can measure and calibrate the radial chromatic aberration of the projector without being affected by the image quality of the camera. The correction effect of this method is that the maximum chromatic aberration of the red-green channel decreases from 1.9591/pixel to 0.5759/pixel, and the average chromatic aberration decreases from 0.2555/pixel to 0.1865/pixel. In addition, blue-green channel maximum chromatic aberration decreased from 1.8906/pixel to 0.5938/pixel, and the average chromatic aberration decreased from 0.2347/pixel to 0.1907/pixel. This method can improve the projection quality for fringe projection 3D profile measurement technology. Full article

(This article belongs to the Special Issue Camera Calibration and 3D Reconstruction)

► Show Figures

Figure 1

24 pages, 4643 KiB

Open AccessArticle

Machine-Learning-Inspired Workflow for Camera Calibration

by Alexey Pak, Steffen Reichel and Jan Burke

Sensors 2022, 22(18), 6804; https://doi.org/10.3390/s22186804 - 8 Sep 2022

Cited by 3 | Viewed by 4001

Abstract

The performance of modern digital cameras approaches physical limits and enables high-precision measurements in optical metrology and in computer vision. All camera-assisted geometrical measurements are fundamentally limited by the quality of camera calibration. Unfortunately, this procedure is often effectively considered a nuisance: calibration [...] Read more.

The performance of modern digital cameras approaches physical limits and enables high-precision measurements in optical metrology and in computer vision. All camera-assisted geometrical measurements are fundamentally limited by the quality of camera calibration. Unfortunately, this procedure is often effectively considered a nuisance: calibration data are collected in a non-systematic way and lack quality specifications; imaging models are selected in an ad hoc fashion without proper justification; and calibration results are evaluated, interpreted, and reported inconsistently. We outline an (arguably more) systematic and metrologically sound approach to calibrating cameras and characterizing the calibration outcomes that is inspired by typical machine learning workflows and practical requirements of camera-based measurements. Combining standard calibration tools and the technique of active targets with phase-shifted cosine patterns, we demonstrate that the imaging geometry of a typical industrial camera can be characterized with sub-mm uncertainty up to distances of a few meters even with simple parametric models, while the quality of data and resulting parameters can be known and controlled at all stages. Full article

(This article belongs to the Special Issue Camera Calibration and 3D Reconstruction)

► Show Figures

Figure 1

14 pages, 1626 KiB

Open AccessArticle

Refocusing Algorithm for Correlation Plenoptic Imaging

by Gianlorenzo Massaro, Francesco V. Pepe and Milena D’Angelo

Sensors 2022, 22(17), 6665; https://doi.org/10.3390/s22176665 - 3 Sep 2022

Cited by 6 | Viewed by 1212

Abstract

Correlation plenoptic imaging (CPI) is a technique capable of acquiring the light field emerging from a scene of interest, namely, the combined information of intensity and propagation direction of light. This is achieved by evaluating correlations between the photon numbers measured by two [...] Read more.

Correlation plenoptic imaging (CPI) is a technique capable of acquiring the light field emerging from a scene of interest, namely, the combined information of intensity and propagation direction of light. This is achieved by evaluating correlations between the photon numbers measured by two high-resolution detectors. Volumetric information about the object of interest is decoded, through data analysis, from the measured four-dimensional correlation function. In this paper, we investigate the relevant aspects of the refocusing algorithm, a post-processing method that isolates the image of a selected transverse plane within the 3D scene, once applied to the correlation function. In particular, we aim at bridging the gap between existing literature, which only deals with refocusing algorithms in case of continuous coordinates, and the experimental reality, in which the correlation function is available as a discrete quantity defined on the sensors pixels. Full article

(This article belongs to the Special Issue Camera Calibration and 3D Reconstruction)

► Show Figures

Figure 1

17 pages, 1900 KiB

Open AccessArticle

Analysis of AI-Based Single-View 3D Reconstruction Methods for an Industrial Application

by Julia Hartung, Patricia M. Dold, Andreas Jahn and Michael Heizmann

Sensors 2022, 22(17), 6425; https://doi.org/10.3390/s22176425 - 25 Aug 2022

Cited by 8 | Viewed by 2626

Abstract

Machine learning (ML) is a key technology in smart manufacturing as it provides insights into complex processes without requiring deep domain expertise. This work deals with deep learning algorithms to determine a 3D reconstruction from a single 2D grayscale image. The potential of [...] Read more.

Machine learning (ML) is a key technology in smart manufacturing as it provides insights into complex processes without requiring deep domain expertise. This work deals with deep learning algorithms to determine a 3D reconstruction from a single 2D grayscale image. The potential of 3D reconstruction can be used for quality control because the height values contain relevant information that is not visible in 2D data. Instead of 3D scans, estimated depth maps based on a 2D input image can be used with the advantage of a simple setup and a short recording time. Determining a 3D reconstruction from a single input image is a difficult task for which many algorithms and methods have been proposed in the past decades. In this work, three deep learning methods, namely stacked autoencoder (SAE), generative adversarial networks (GANs) and U-Nets are investigated, evaluated and compared for 3D reconstruction from a 2D grayscale image of laser-welded components. In this work, different variants of GANs are tested, with the conclusion that Wasserstein GANs (WGANs) are the most robust approach among them. To the best of our knowledge, the present paper considers for the first time the U-Net, which achieves outstanding results in semantic segmentation, in the context of 3D reconstruction tasks. Unlike the U-Net, which uses standard convolutions, the stacked dilated U-Net (SDU-Net) applies stacked dilated convolutions. Of all the 3D reconstruction approaches considered in this work, the SDU-Net shows the best performance, not only in terms of evaluation metrics but also in terms of computation time. Due to the comparably small number of trainable parameters and the suitability of the architecture for strong data augmentation, a robust model can be generated with only a few training data. Full article

(This article belongs to the Special Issue Camera Calibration and 3D Reconstruction)

► Show Figures

Figure 1

15 pages, 4613 KiB

Open AccessArticle

Monocular Depth Estimation: Lightweight Convolutional and Matrix Capsule Feature-Fusion Network

by Yinchu Wang and Haijiang Zhu

Sensors 2022, 22(17), 6344; https://doi.org/10.3390/s22176344 - 23 Aug 2022

Cited by 1 | Viewed by 1746

Abstract

This paper reports a study that aims to solve the problem of the weak adaptability to angle transformation of current monocular depth estimation algorithms. These algorithms are based on convolutional neural networks (CNNs) but produce results lacking in estimation accuracy and robustness. The [...] Read more.

This paper reports a study that aims to solve the problem of the weak adaptability to angle transformation of current monocular depth estimation algorithms. These algorithms are based on convolutional neural networks (CNNs) but produce results lacking in estimation accuracy and robustness. The paper proposes a lightweight network based on convolution and capsule feature fusion (CNNapsule). First, the paper introduces a fusion block module that integrates CNN features and matrix capsule features to improve the adaptability of the network to perspective transformations. The fusion and deconvolution features are fused through skip connections to generate a depth image. In addition, the corresponding loss function is designed according to the long-tail distribution, gradient similarity, and structural similarity of the datasets. Finally, the results are compared with the methods applied to the NYU Depth V2 and KITTI datasets and show that our proposed method has better accuracy on the C1 and C2 indices and a better visual effect than traditional methods and deep learning methods without transfer learning. The number of trainable parameters required by this method is 65% lower than that required by methods presented in the literature. The generalization of this method is verified via the comparative testing of the data collected from the internet and mobile phones. Full article

(This article belongs to the Special Issue Camera Calibration and 3D Reconstruction)

► Show Figures

Figure 1

23 pages, 5833 KiB

Open AccessArticle

A Wind Tunnel Setup for Fluid-Structure Interaction Measurements Using Optical Methods

by Simon Nietiedt, Tom T. B. Wester, Apostolos Langidis, Lars Kröger, Robin Rofallski, Martina Göring, Martin Kühn, Gerd Gülker and Thomas Luhmann

Sensors 2022, 22(13), 5014; https://doi.org/10.3390/s22135014 - 2 Jul 2022

Cited by 4 | Viewed by 2201

Abstract

The design of rotor blades is based on information about aerodynamic phenomena. An important one is fluid-structure interaction (FSI) which describes the interaction between a flexible object (rotor blade) and the surrounding fluid (wind). However, the acquisition of FSI is complex, and only [...] Read more.

The design of rotor blades is based on information about aerodynamic phenomena. An important one is fluid-structure interaction (FSI) which describes the interaction between a flexible object (rotor blade) and the surrounding fluid (wind). However, the acquisition of FSI is complex, and only a few practical concepts are known. This paper presents a measurement setup to acquire real information about the FSI of rotating wind turbines in wind tunnel experiments. The setup consists of two optical measurement systems to simultaneously record fluid (PIV system) and deformation (photogrammetry system) information in one global coordinate system. Techniques to combine both systems temporally and spatially are discussed in this paper. Furthermore, the successful application is shown by several experiments. Here, different wind conditions are applied. The experiments show that the new setup can acquire high-quality area-based information about fluid and deformation. Full article

(This article belongs to the Special Issue Camera Calibration and 3D Reconstruction)

► Show Figures

Figure 1

16 pages, 5128 KiB

Open AccessArticle

Microsoft Azure Kinect Calibration for Three-Dimensional Dense Point Clouds and Reliable Skeletons

by Laura Romeo, Roberto Marani, Anna Gina Perri and Tiziana D’Orazio

Sensors 2022, 22(13), 4986; https://doi.org/10.3390/s22134986 - 1 Jul 2022

Cited by 4 | Viewed by 2801

Abstract

Nowadays, the need for reliable and low-cost multi-camera systems is increasing for many potential applications, such as localization and mapping, human activity recognition, hand and gesture analysis, and object detection and localization. However, a precise camera calibration approach is mandatory for enabling further [...] Read more.

Nowadays, the need for reliable and low-cost multi-camera systems is increasing for many potential applications, such as localization and mapping, human activity recognition, hand and gesture analysis, and object detection and localization. However, a precise camera calibration approach is mandatory for enabling further applications that require high precision. This paper analyzes the available two-camera calibration approaches to propose a guideline for calibrating multiple Azure Kinect RGB-D sensors to achieve the best alignment of point clouds in both color and infrared resolutions, and skeletal joints returned by the Microsoft Azure Body Tracking library. Different calibration methodologies using 2D and 3D approaches, all exploiting the functionalities within the Azure Kinect devices, are presented. Experiments demonstrate that the best results are returned by applying 3D calibration procedures, which give an average distance between all couples of corresponding points of point clouds in color or an infrared resolution of 21.426 mm and 9.872 mm for a static experiment and of 20.868 mm and 7.429 mm while framing a dynamic scene. At the same time, the best results in body joint alignment are achieved by three-dimensional procedures on images captured by the infrared sensors, resulting in an average error of 35.410 mm. Full article

(This article belongs to the Special Issue Camera Calibration and 3D Reconstruction)

► Show Figures

Figure 1

27 pages, 29825 KiB

Open AccessArticle

Noise-Resilient Depth Estimation for Light Field Images Using Focal Stack and FFT Analysis

by Rishabh Sharma, Stuart Perry and Eva Cheng

Sensors 2022, 22(5), 1993; https://doi.org/10.3390/s22051993 - 3 Mar 2022

Cited by 1 | Viewed by 2379

Abstract

Depth estimation for light field images is essential for applications such as light field image compression, reconstructing perspective views and 3D reconstruction. Previous depth map estimation approaches do not capture sharp transitions around object boundaries due to occlusions, making many of the current [...] Read more.

Depth estimation for light field images is essential for applications such as light field image compression, reconstructing perspective views and 3D reconstruction. Previous depth map estimation approaches do not capture sharp transitions around object boundaries due to occlusions, making many of the current approaches unreliable at depth discontinuities. This is especially the case for light field images because the pixels do not exhibit photo-consistency in the presence of occlusions. In this paper, we propose an algorithm to estimate the depth map for light field images using depth from defocus. Our approach uses a small patch size of pixels in each focal stack image for comparing defocus cues, allowing the algorithm to generate sharper depth boundaries. Then, in contrast to existing approaches that use defocus cues for depth estimation, we use frequency domain analysis image similarity checking to generate the depth map. Processing in the frequency domain reduces the individual pixel errors that occur while directly comparing RGB images, making the algorithm more resilient to noise. The algorithm has been evaluated on both a synthetic image dataset and real-world images in the JPEG dataset. Experimental results demonstrate that our proposed algorithm outperforms state-of-the-art depth estimation techniques for light field images, particularly in case of noisy images. Full article

(This article belongs to the Special Issue Camera Calibration and 3D Reconstruction)

► Show Figures

Figure 1

16 pages, 3431 KiB

Open AccessArticle

Virtual Namesake Point Multi-Source Point Cloud Data Fusion Based on FPFH Feature Difference

by Li Zheng and Zhukun Li

Sensors 2021, 21(16), 5441; https://doi.org/10.3390/s21165441 - 12 Aug 2021

Cited by 15 | Viewed by 2737

Abstract

There are many sources of point cloud data, such as the point cloud model obtained after a bundle adjustment of aerial images, the point cloud acquired by scanning a vehicle-borne light detection and ranging (LiDAR), the point cloud acquired by terrestrial laser scanning, [...] Read more.

There are many sources of point cloud data, such as the point cloud model obtained after a bundle adjustment of aerial images, the point cloud acquired by scanning a vehicle-borne light detection and ranging (LiDAR), the point cloud acquired by terrestrial laser scanning, etc. Different sensors use different processing methods. They have their own advantages and disadvantages in terms of accuracy, range and point cloud magnitude. Point cloud fusion can combine the advantages of each point cloud to generate a point cloud with higher accuracy. Following the classic Iterative Closest Point (ICP), a virtual namesake point multi-source point cloud data fusion based on Fast Point Feature Histograms (FPFH) feature difference is proposed. For the multi-source point cloud with noise, different sampling resolution and local distortion, it can acquire better registration effect and improve the accuracy of low precision point cloud. To find the corresponding point pairs in the ICP algorithm, we use the FPFH feature difference, which can combine surrounding neighborhood information and have strong robustness to noise, to generate virtual points with the same name to obtain corresponding point pairs for registration. Specifically, through the establishment of voxels, according to the F2 distance of the FPFH of the target point cloud and the source point cloud, the convolutional neural network is used to output a virtual and more realistic and theoretical corresponding point to achieve multi-source Point cloud data registration. Compared with the ICP algorithm for finding corresponding points in existing points, this method is more reasonable and more accurate, and can accurately correct low-precision point cloud in detail. The experimental results show that the accuracy of our method and the best algorithm is equivalent under the clean point cloud and point cloud of different resolutions. In the case of noise and distortion in the point cloud, our method is better than other algorithms. For low-precision point cloud, it can better match the target point cloud in detail, with better stability and robustness. Full article

(This article belongs to the Special Issue Camera Calibration and 3D Reconstruction)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Camera Calibration and 3D Reconstruction

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Published Papers (11 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI