Search Results (156)

The Surface Extraction by TIN-based Search-space Minimization (SETSM) algorithm provides automatic generation of stereo-photogrammetric Digital Surface Models (DSMs) from single stereopairs of stereoscopic images (i.e., stereopairs), eliminating the need for terrain-dependent parameters. SETSM has been extensively validated through the ArcticDEM and Reference Elevation Models for Antarctica (REMA) DSM mapping projects. To enhance DSM coverage, quality, and accuracy by addressing stereopair occlusions, we expand the capabilities of the SETSM algorithm from single stereopair to multiple-pair matching. Building on SETSM’s essential components, we present a SETSM multiple-pair matching procedure (SETSM MMP) that modifies 3D voxel construction, similarity measurement, and blunder detection, among other components. A novel Three-Dimensional Kernel-based Weighted Height Estimation (3D KWHE) algorithm specialized for SETSM accurately determines optimal heights and reduces surface noise. Additionally, an adaptive pixel-to-pixel matching strategy mitigates the effect of differences in ground sample distance (GSD) between images. Validation using space-borne Worldview-2 and air-borne DMC multiple images over urban landscapes, compared to USGS lidar DSM, confirms improved height accuracy and matching success rates. The results from the DMC air-borne images demonstrate efficient elimination of occlusions. SETSM MMP enables high-quality DSM generation in urban environments while retaining the original, single-stereopair SETSM’s high performance. Full article

Dynamic deformation measurement (DDM) is critical across infrastructure and industrial applications. Among various advanced techniques, multi-camera digital image correlation (MC-DIC) stands out due to its ability to achieve wide-range, full-field, and non-contact 3D DDM by pairing camera subsystems. However, existing MC-DIC methods typically rely on inefficient manual pairing or a simplistic strategy that aggregates all visible cameras for measuring specific object regions, leading to camera over-grouping. These limitations often result in cumbersome system setup and ill-measured deformations. To overcome these challenges, we propose a novel MC-DIC method with pointwise-optimized model-based stereo pairing (MPMC-DIC). By automatically evaluating and selecting camera pairs based on five evaluation factors derived from 3D model and calibrated cameras, the proposed method overcomes the over-grouping problem and achieves high-precision DDM of semi-rigid objects. A Ø5 × 5 cm cylinder experiment demonstrated an accuracy of 0.03 mm for both horizontal and depth displacements in the 0.0–5.0 mm range, and validated strong robustness against cluttered backgrounds using a 2 × 4 camera array. Vibration measurement of a 9 × 15 × 16 cm PC speaker operating at 50 Hz, using eight surrounding cameras capturing 1920 × 1080 images at 400 fps, confirmed the proposed method’s capability to perform wide-range dynamic deformation analysis and its robustness against complex object geometries. Full article

Robust and accurate line segment matching remains a critical challenge in stereo vision, particularly in space-based applications where weak texture, structural symmetry, and strong illumination variations are common. This paper presents a multi-constraint progressive matching framework that integrates epipolar geometry, coplanarity verification, local homography, angular consistency, and distance-ratio invariance to establish reliable line correspondences. A unified cost matrix is constructed by quantitatively encoding these geometric residuals, enabling comprehensive candidate evaluation. To ensure global consistency and suppress mismatches, the final assignment is optimized using a Hungarian algorithm under one-to-one matching constraints. Extensive experiments on a wide range of stereo image pairs demonstrate that the proposed method consistently outperforms several advanced conventional approaches in terms of accuracy, robustness, and computational efficiency, as evidenced by both quantitative and qualitative evaluations. Full article

Multi-camera Visual Simultaneous Localization and Mapping (V-SLAM) increases spatial coverage through multi-view image streams, improving localization accuracy and reducing data acquisition time. Despite its speed and generally robustness, V-SLAM often struggles to achieve precise camera poses necessary for accurate 3D reconstruction, especially in complex environments. This study introduces two novel multi-camera optimization methods to enhance pose accuracy, reduce drift, and ensure loop closures. These methods refine multi-camera V-SLAM outputs within existing frameworks and are evaluated in two configurations: (1) multiple independent stereo V-SLAM instances operating on separate camera pairs; and (2) multi-view odometry processing all camera streams simultaneously. The proposed optimizations include (1) a multi-view feature-based optimization that integrates V-SLAM poses with rigid inter-camera constraints and bundle adjustment; and (2) a multi-camera pose graph optimization that fuses multiple trajectories using relative pose constraints and robust noise models. Validation is conducted through two complex 3D surveys using the ATOM-ANT3D multi-camera fisheye mobile mapping system. Results demonstrate survey-grade accuracy comparable to traditional photogrammetry, with reduced computational time, advancing toward near real-time 3D mapping of challenging environments. Full article

Synthetic Aperture Radar (SAR) images and optical imagery have high potential for extracting digital elevation models (DEMs). The two main approaches for deriving elevation models from SAR data are interferometry (InSAR) and radargrammetry. Adapted from photogrammetric principles, radargrammetry relies on disparity model estimation as its core component. Matching strategies in radargrammetry typically follow local, global, or semi-global methodologies. Local methods, while having higher accuracy, especially in low-texture SAR images, require larger kernel sizes, leading to quadratic computational complexity. Conversely, global and semi-global models produce more consistent and higher-quality disparity maps but are computationally more intensive than local methods with small kernels and require more memory (RAM). In this study, inspired by the advantages of local matching algorithms, a computationally efficient and novel model is proposed for extracting corresponding pixels in SAR-intensity stereo images. To enhance accuracy, the proposed two-stage algorithm operates without an image pyramid structure. Notably, unlike traditional local and global models, the computational complexity of the proposed approach remains stable as the input size or kernel dimensions increase while memory consumption stays low. Compared to a pyramid-based local normalized cross-correlation (NCC) algorithm and adaptive semi-global matching (SGM) models, the proposed method maintains good accuracy comparable to adaptive SGM while reducing processing time by up to 50% relative to pyramid SGM and achieving a 35-fold speedup over the local NCC algorithm with an optimal kernel size. Validated on a Sentinel-1 stereo pair with a 10 m ground-pixel size, the proposed algorithm yields a DEM with an average accuracy of 34.1 m. Full article

We present HGC-Net, a hybrid pipeline for assessing geometric consistency between stereo image pairs. Our method integrates classical epipolar geometry with deep learning components to compute an interpretable scalar score A, reflecting the degree of alignment. Unlike traditional techniques, which may overlook subtle miscalibrations, HGC-Net reliably detects both severe and mild geometric distortions, such as sub-degree tilts and pixel-level shifts. We evaluate the method on the Middlebury 2014 stereo dataset, using synthetically distorted variants to simulate misalignments. Experimental results show that our score degrades smoothly with increasing geometric error and achieves high detection rates even at minimal distortion levels, outperforming baseline approaches based on disparity or calibration checks. The method operates in real time (12.5 fps on 1080p input) and does not require access to internal camera parameters, making it suitable for embedded stereo systems and quality monitoring in robotic and AR/VR applications. The approach also supports explainability via confidence maps and anomaly heatmaps, aiding human operators in identifying problematic regions. Full article

Located on the northern side of the China–Pakistan Highway in the Pamir Plateau, Laqi Gully represents a typical rainfall–meltwater coupled debris flow gully. During 2020–2024, seven debris flow events occurred in this area, four of which disrupted traffic and posed significant threats to the China–Pakistan Economic Corridor (CPEC). The hazard assessment of debris flows constitutes a crucial component in disaster prevention and mitigation. However, current research presents two critical limitations: traditional models primarily focus on single precipitation-driven debris flows, while low-resolution digital elevation models (DEMs) inadequately characterize the topographic features of alpine narrow valleys. Addressing these issues, this study employed GF-7 satellite stereo image pairs to construct a 1 m resolution DEM and systematically simulated debris flow propagation processes under 10–100-year recurrence intervals using a coupled rainfall–meltwater model. The results show the following: (1) The mudslide develops rapidly in the gully section, and the flow velocity decays when it reaches the highway. (2) At highway cross-sections, maximum velocities corresponding to 10-, 20-, 50-, and 100-year recurrence intervals measure 2.57 m/s, 2.75 m/s, 3.02 m/s, and 3.36 m/s, respectively, with maximum flow depths of 1.56 m, 1.78 m, 2.06 m, and 2.52 m. (3) Based on the hazard classification model of mudslide intensity and return period, the high-, medium-, and low-hazard sections along the highway were 58.65 m, 27.36 m, and 24.1 m, respectively. This research establishes a novel hazard assessment methodology for rainfall–meltwater coupled debris flows in narrow valleys, providing technical support for debris flow mitigation along the CPEC. The outcomes demonstrate significant practical value for advancing infrastructure sustainability under the United Nations Sustainable Development Goals (SDGs). Full article

Advancements in remote sensing technology have remarkably enhanced the 3D Earth surface reconstruction, which is pivotal for applications such as disaster relief, emergency management, and urban planning, etc. Although satellite imagery offers a cost-effective and extensive coverage solution for 3D reconstruction, the quality of the resulted digital surface model (DSM) heavily relies on the choice of stereo image pairs. However, current approaches of stereo Earth observation still employ a post-acquisition manner without sophisticated planning in advance, causing inefficiencies and low reconstruction quality. This paper introduces a novel quality-driven planning method for satellite stereo imaging, aiming at optimizing the search of stereo pairs to achieve high-quality 3D reconstruction. Moreover, a regression model is customized and incorporated to estimate the reconstructed point cloud geopositioning quality, based on the enhanced features of possible Earth-imaging opportunities. Experiments conducted on both real satellite images and simulated constellation data demonstrate the efficacy of the proposed method in estimating reconstruction quality beforehand and searching for optimal stereo pair combinations as the final satellite imaging schedule, which can improve the stereo quality significantly. Full article

With the advancement of agricultural automation technologies, apple-harvesting robots have gradually become a focus of research. As their “perceptual core,” machine vision systems directly determine picking success rates and operational efficiency. However, existing vision systems still exhibit significant shortcomings in target detection and positioning accuracy in complex orchard environments (e.g., uneven illumination, foliage occlusion, and fruit overlap), which hinders practical applications. This study proposes a visual system for apple-harvesting robots based on improved Mask R-CNN and binocular vision to achieve more precise fruit positioning. The binocular camera (ZED2i) carried by the robot acquires dual-channel apple images. An improved Mask R-CNN is employed to implement instance segmentation of apple targets in binocular images, followed by a template-matching algorithm with parallel epipolar constraints for stereo matching. Four pairs of feature points from corresponding apples in binocular images are selected to calculate disparity and depth. Experimental results demonstrate average coefficients of variation and positioning accuracy of 5.09% and 99.61%, respectively, in binocular positioning. During harvesting operations with a self-designed apple-picking robot, the single-image processing time was 0.36 s, the average single harvesting cycle duration reached 7.7 s, and the comprehensive harvesting success rate achieved 94.3%. This work presents a novel high-precision visual positioning method for apple-harvesting robots. Full article

The use of high-resolution satellite stereo pairs for dense image matching is a core technology for the low-cost generation of large-scale digital surface models (DSMs). However, water areas in satellite imagery often exhibit weak texture characteristics. This leads to serious issues in reconstructing water surface DSMs with traditional dense matching methods, such as significant holes and abnormal undulations. These problems significantly impact the intelligent application of satellite DSM products. To address these issues, this study innovatively proposes a water region DSM reconstruction method, boundary plane-constrained surface water stereo reconstruction (BPC-SWSR). The algorithm constructs a water surface reconstruction model with constraints on the plane’s tilt angle and boundary, combining effective ground matching data from the shoreline and the plane constraints of the water surface. This method achieves the seamless planar reconstruction of the water region, effectively solving the technical challenges of low geometric accuracy in water surface DSMs. This article conducts experiments on 10 high-resolution satellite stereo image pairs, covering three types of water bodies: river, lake, and sea. Ground truth water surface elevations were obtained through a manual tie point selection followed by forward intersection and planar fitting in water surface areas, establishing a rigorous validation framework. The DSMs generated by the proposed algorithm were compared with those generated by state-of-the-art dense matching algorithms and the industry-leading software Reconstruction Master version 6.0. The proposed algorithm achieves a mean RMSE of 2.279 m and a variance of 0.6613 m² in water surface elevation estimation, significantly outperforming existing methods with average RMSE and a variance of 229.2 m and 522.5 m², respectively. This demonstrates the algorithm’s ability to generate more accurate and smoother water surface models. Furthermore, the algorithm still achieves excellent reconstruction results when processing different types of water areas, confirming its wide applicability in real-world scenarios. Full article

Stereo images, which consist of left and right image pairs, are often unaligned when initially captured, as they represent raw data. Stereo images are typically used in scenarios requiring disparity between the left and right views, such as depth estimation. In such cases, image calibration is performed to obtain the necessary parameters, and, based on these parameters, image rectification is applied to align the epipolar lines of the stereo images. This preprocessing step is crucial for effectively utilizing stereo images. The conventional method for performing image calibration usually involves using a reference object, such as a checkerboard, to obtain these parameters. In this paper, we propose a novel approach that does not require any special reference points like a checkerboard. Instead, we employ object detection to segment object pairs and calculate the centroids of the segmented objects. By aligning the y-coordinates of these centroids in the left and right image pairs, we induce the epipolar lines to be parallel, achieving an effect similar to image rectification. Full article

Agricultural robots can mitigate labor shortages and advance precision farming. However, the dense vegetation canopies and uneven terrain in orchard environments reduce the reliability of traditional GPS-based localization, thereby reducing navigation accuracy and making autonomous navigation challenging. Moreover, inefficient path planning and an increased risk of collisions affect the robot’s ability to perform tasks such as fruit harvesting, spraying, and monitoring. To address these limitations, this study integrated stereo visual odometry with real-time appearance-based mapping (RTAB-Map)-based simultaneous localization and mapping (SLAM) to improve mapping and localization in both indoor and outdoor orchard settings. The proposed system leverages stereo image pairs for precise depth estimation while utilizing RTAB-Map’s graph-based SLAM framework with loop-closure detection to ensure global map consistency. In addition, an incorporated inertial measurement unit (IMU) enhances pose estimation, thereby improving localization accuracy. Substantial improvements in both mapping and localization performance over the traditional approach were demonstrated, with an average error of 0.018 m against the ground truth for outdoor mapping and a consistent average error of 0.03 m for indoor trails with a 20.7% reduction in visual odometry trajectory deviation compared to traditional methods. Localization performance remained robust across diverse conditions, with a low RMSE of 0.207 m. Our approach provides critical insights into developing more reliable autonomous navigation systems for agricultural robots. Full article

This paper provides a comparative analysis of calibration techniques applicable to stereo and near-infrared (NIR) camera systems, with a specific emphasis on the Intel RealSense SR300 alongside a standard 2-megapixel NIR camera. This study investigates the pivotal function of calibration within both stereo vision and NIR imaging applications, which are essential across various domains, including robotics, augmented reality, and low-light imaging. For stereo systems, we scrutinise the conventional method involving a 9 × 6 chessboard pattern utilised to ascertain the intrinsic and extrinsic camera parameters. The proposed methodology consists of three main steps: (1) real-time calibration error classification for stereo cameras, (2) NIR-specific calibration techniques, and (3) a comprehensive evaluation framework. This research introduces a novel real-time evaluation methodology that classifies calibration errors predicated on the pixel offsets between corresponding points in the left and right images. Conversely, NIR camera calibration techniques are modified to address the distinctive properties of near-infrared light. We deliberate on the difficulties encountered in devising NIR–visible calibration patterns and the imperative to consider the spectral response and temperature sensitivity within the calibration procedure. The paper also puts forth an innovative calibration assessment application that is relevant to both systems. Stereo cameras evaluate the corner detection accuracy in real time across multiple image pairs, whereas NIR cameras concentrate on assessing the distortion correction and intrinsic parameter accuracy under varying lighting conditions. Our experiments validate the necessity of routine calibration assessment, as environmental factors may compromise the calibration quality over time. We conclude by underscoring the disparities in the calibration requirements between stereo and NIR systems, thereby emphasising the need for specialised approaches tailored to each domain to guarantee an optimal performance in their respective applications. Full article

Three-dimensional reconstruction entails the development of mathematical models of three-dimensional objects that are suitable for computational representation and processing. This technique constructs realistic 3D models of images and has significant practical applications across various fields. This study proposes a rapid and precise multi-view 3D reconstruction method to address the challenges of low reconstruction efficiency and inadequate, poor-quality point cloud generation in incremental structure-from-motion (SFM) algorithms in multi-view geometry. The methodology involves capturing a series of overlapping images of campus. We employed the Scale-invariant feature transform (SIFT) algorithm to extract feature points from each image, applied the KD-Tree algorithm for inter-image matching, and Enhanced autonomous threshold adjustment by utilizing the Random sample consensus (RANSAC) algorithm to eliminate mismatches, thereby enhancing feature matching accuracy and the number of matched point pairs. Additionally, we developed a feature-matching strategy based on similarity, which optimizes the pairwise matching process within the incremental structure from a motion algorithm. This approach decreased the number of matches and enhanced both algorithmic efficiency and model reconstruction accuracy. For dense reconstruction, we utilized the patch-based multi-view stereo (PMVS) algorithm, which is based on facets. The results indicate that our proposed method achieves a higher number of reconstructed feature points and significantly enhances algorithmic efficiency by approximately ten times compared to the original incremental reconstruction algorithm. Consequently, the generated point cloud data are more detailed, and the textures are clearer, demonstrating that our method is an effective solution for three-dimensional reconstruction. Full article

Multispectral imaging plays a key role in crop monitoring. A major challenge, however, is spectral band misalignment, which can hinder accurate plant health assessment by distorting the calculation of vegetation indices. This study presents a novel approach for short-range calibration of a multispectral camera, utilizing stereo vision for precise geometric correction of acquired images. By using multispectral camera lenses as binocular pairs, the sensor acquisition distance was estimated, and an alignment model was developed for distances ranging from 500 mm to 1500 mm. The approach relied on selecting the red band image as a reference, while the remaining bands were treated as moving images. The stereo camera calibration algorithm estimated the target distance, enabling the correction of band misalignment through previously developed models. The alignment models were applied to assess the health status of baby leaf crops (Lactuca sativa cv. Maverik) by analyzing spectral indices correlated with chlorophyll content. The results showed that the stereo vision approach used for distance estimation achieved high accuracy, with average reprojection errors of approximately 0.013 pixels (4.485 × 10⁻⁵ mm). Additionally, the proposed linear model was able to explain reasonably the effect of distance on alignment offsets. The overall performance of the proposed experimental alignment models was satisfactory, with offset errors on the bands less than 3 pixels. Despite the results being not yet sufficiently robust for a fully predictive model of chlorophyll content in plants, the analysis of vegetation indices demonstrated a clear distinction between healthy and unhealthy plants. Full article