Search Results (27)

The real-time detection and localization of dynamic targets in cattle farms are crucial for the effective operation of intelligent equipment. To overcome the limitations of wearable devices, including high costs and operational stress, this paper proposes a lightweight, non-contact solution. The goal is to improve the accuracy and efficiency of target localization while reducing the complexity of the system. A novel approach is introduced based on YOLOv8s, incorporating a C2f_DW_StarBlock module. The system fuses binocular images from a ZED2i camera with GPS and IMU data to form a multimodal ranging and localization module. Experimental results demonstrate a 36.03% reduction in model parameters, a 33.45% decrease in computational complexity, and a 38.67% reduction in model size. The maximum ranging error is 4.41%, with localization standard deviations of 1.02 m (longitude) and 1.10 m (latitude). The model is successfully integrated into an ROS system, achieving stable real-time performance. This solution offers the advantages of being lightweight, non-contact, and low-maintenance, providing strong support for intelligent farm management and multi-target monitoring. Full article

This paper presents a novel approach to determine dynamic safety and comfort zones in human–robot interactions (HRIs), with a focus on service robots operating in dynamic environments with people. The proposed proxemic model leverages the golden ratio-based comfort zone distribution and ISO safety standards to define adaptive proxemic boundaries for robots around humans. Unlike traditional fixed-threshold approaches, this novel method proposes a gradual and context-sensitive modulation of robot behaviour based on human position, orientation, and relative velocity. The system was implemented on an NVIDIA Jetson Xavier NX platform using a ZED 2i stereo depth camera Stereolabs, New York, USA and tested on two mobile robotic platforms: Go1 Unitree, Hangzhou, China (quadruped) and Scout Mini Agilex, Dongguan, China (wheeled). The initial verification of proposed proxemic model through experimental comfort validation was conducted using two simple interaction scenarios, and subjective feedback was collected from participants using a modified Godspeed Questionnaire Series. The results show that the participants felt comfortable during the experiments with robots. This acceptance of the proposed methodology plays an initial role in supporting further research of the methodology. The proposed solution also facilitates integration into existing navigation frameworks and opens pathways towards socially aware robotic systems. Full article

With the advancement of agricultural automation technologies, apple-harvesting robots have gradually become a focus of research. As their “perceptual core,” machine vision systems directly determine picking success rates and operational efficiency. However, existing vision systems still exhibit significant shortcomings in target detection and positioning accuracy in complex orchard environments (e.g., uneven illumination, foliage occlusion, and fruit overlap), which hinders practical applications. This study proposes a visual system for apple-harvesting robots based on improved Mask R-CNN and binocular vision to achieve more precise fruit positioning. The binocular camera (ZED2i) carried by the robot acquires dual-channel apple images. An improved Mask R-CNN is employed to implement instance segmentation of apple targets in binocular images, followed by a template-matching algorithm with parallel epipolar constraints for stereo matching. Four pairs of feature points from corresponding apples in binocular images are selected to calculate disparity and depth. Experimental results demonstrate average coefficients of variation and positioning accuracy of 5.09% and 99.61%, respectively, in binocular positioning. During harvesting operations with a self-designed apple-picking robot, the single-image processing time was 0.36 s, the average single harvesting cycle duration reached 7.7 s, and the comprehensive harvesting success rate achieved 94.3%. This work presents a novel high-precision visual positioning method for apple-harvesting robots. Full article

This paper presents a self-driving system for automatic parallel parking, integrating obstacle avoidance for enhanced safety. The vehicle platform employs three primary sensors—a web camera, a Zed depth camera, and LiDAR—to perceive its surroundings, including sidewalks and potential obstacles. By processing camera and LiDAR data, the system determines the vehicle’s position and assesses parking space availability, with LiDAR also aiding in malfunction detection. The system operates in three stages: parking space identification, path planning using geometric circles, and fine-tuning with fuzzy control if misalignment is detected. Experimental results, evaluated visually in a model-scale setup, confirm the system’s ability to achieve smooth and reliable parallel parking maneuvers. Quantitative performance metrics, such as precise parking accuracy or total execution time, were not recorded in this study but will be included in future work to further support the system’s effectiveness. Full article

In this research, I introduce a water surface elevation measurement method that combines point cloud processing techniques and stereo vision cameras. While current vision-based water level measurement techniques focus on laboratory measurements or are based on auxiliary devices such as water rulers, I investigated the feasibility of measuring elevation based on images of the water surface. This research implements a monitoring system on-site, comprising a ZED 2i binocular camera (Stereolabs, San Francisco, CA, USA). First, the uncertainty of the camera is evaluated in a real measurement scenario. Then, the water surface images captured by the binocular camera are stereo matched to obtain parallax maps. Subsequently, the results of the binocular camera calibration are utilized to obtain the 3D point cloud coordinate values of the water surface image. Finally, the horizontal plane equation is solved by the RANSAC algorithm to finalize the height of the camera on the water surface. This approach is particularly significant as it offers a non-contact, shore-based solution that eliminates the need for physical water references, thereby enhancing the adaptability and efficiency of water level monitoring in challenging environments, such as remote or inaccessible areas. Within a measured elevation of 5 m, the water level measurement error is less than 2 cm. Full article

In this research, we present a novel approach for cone clustering, path planning, and path visualization in autonomous Formula Student race cars, utilizing the YOLOv8 model and a ZED 2 camera, executed on a Jetson Orin computer. Our system first identifies and then deprojects the positions of cones in space, employing an advanced clustering mechanism to generate midpoints and draw connecting lines. In previous clustering algorithms, cones were stored separately by color and connected based on relevance to create the lane edges. However, our proposed solution adopts a fundamentally different approach. Cones on the left and right sides within a dynamically changing maximum and minimum distance are connected by a central line, and the midpoint of this line is marked distinctly. Cones connected in this manner are then linked by their positions to form the edges of the track. The midpoints on these central lines are displayed as markers, facilitating the visualization of the optimal path. In our research, we also cover the analysis of the clustering algorithm on global maps. The implementation utilizes the ROS 2 framework for real-time data handling and visualization. Our results demonstrate the system’s efficiency in dynamic environments, highlighting potential advancements in the field of autonomous racing. The limitation of our approach is the dependency on precise cone detection and classification, which may be affected by environmental factors such as lighting and cone positioning. Full article

In this paper, we consider the use of high-definition maps for autonomous vehicle (AV) localization. An autonomous vehicle may have a variety of sensors, including cameras, lidars, and Global Positioning System(GPS) sensors. Each sensor technology has its own pros and cons; for example, GPS may not be very effective in a city environment with high-rise buildings; cameras may not be very effective in poorly illuminated environments; and lidars simply generate a relatively dense local point cloud. In a typical autonomous vehicle system, all of these sensors are present and sensor fusion algorithms are used to extract the most accurate information. Using our AV research vehicle, we drove on our university campus and recorded Real Time Kinematic-GPS(RTK-GPS) (ZED-F9P) and Velodyne Lidar (VLP-16) data in a time-synchronized fashion. In other words, for every GPS location on our campus, we have lidar-generated point cloud data, resulting in a simple high-definition map of the campus. The main challenge that we look to overcome in this paper is thus: given a high-definition map of the environment and local point cloud data generated by a single lidar scan, determine the AV research vehicle’s location by using point cloud “similarity” metrics. We first propose a computationally simple similarity metric and then describe a recursive Kalman filter-like approach for localization. The effectiveness of the proposed similarity metric has been demonstrated using the experimental data. Full article

Cattle farming is an important part of the global livestock industry, and cattle body size is the key indicator of livestock growth. However, traditional manual methods for measuring body sizes are not only time-consuming and labor-intensive but also incur significant costs. Meanwhile, automatic measurement techniques are prone to being affected by environmental conditions and the standing postures of livestock. To overcome these challenges, this study proposes a multi-view fusion-driven automatic measurement system for full-attitude cattle body measurements. Outdoors in natural light, three Zed2 cameras were installed covering different views of the channel. Multiple images, including RGB images, depth images, and point clouds, were automatically acquired from multiple views using the YOLOv8n algorithm. The point clouds from different views undergo multiple denoising to become local point clouds of the cattle body. The local point clouds are coarsely and finely aligned to become a complete point cloud of the cattle body. After detecting the 2D key points on the RGB image created by the YOLOv8x-pose algorithm, the 2D key points are mapped onto the 3D cattle body by combining the internal parameters of the camera and the depth values of the corresponding pixels of the depth map. Based on the mapped 3D key points, the body sizes of cows in different poses are automatically measured, including height, length, abdominal circumference, and chest circumference. In addition, support vector machines and Bézier curves are employed to rectify the missing and deformed circumference body sizes caused by environmental effects. The automatic body measurement system measured the height, length, abdominal circumference, and chest circumference of 47 Huaxi Beef Cattle, a breed native to China, and compared the results with manual measurements. The average relative errors were 2.32%, 2.27%, 3.67%, and 5.22%, respectively, when compared with manual measurements, demonstrating the feasibility and accuracy of the system. Full article

The rapid growth of the real estate market has led to the appearance of more and more residential areas and large apartment buildings that need to be managed and maintained by a single real estate developer or company. This scientific article details the development of a novel method for inspecting buildings in a semi-automated manner, thereby reducing the time needed to assess the requirements for the maintenance of a building. This paper focuses on the development of an application which has the purpose of detecting imperfections in a range of building sections using a combination of machine learning techniques and 3D scanning methodologies. This research focuses on the design and development of a machine learning-based application that utilizes the Python programming language and the PyTorch library; it builds on the team′s previous study, in which they investigated the possibility of applying their expertise in creating construction-related applications for real-life situations. Using the Zed camera system, real-life pictures of various building components were used, along with stock images when needed, to train an artificial intelligence model that could identify surface damage or defects such as cracks and differentiate between naturally occurring elements such as shadows or stains. One of the goals is to develop an application that can identify defects in real time while using readily available tools in order to ensure a practical and affordable solution. The findings of this study have the potential to greatly enhance the availability of defect detection procedures in the construction sector, which will result in better building maintenance and structural integrity. Full article

The condition of the roads where cars circulate is of the utmost importance to ensure that each autonomous or manual car can complete its journey satisfactorily. The existence of potholes, speed bumps, and other irregularities in the pavement can cause car wear and fatal traffic accidents. Therefore, detecting and characterizing these anomalies helps reduce the risk of accidents and damage to the vehicle. However, street images are naturally multivariate, with redundant and substantial information, as well as significantly contaminated measurement noise, making the detection of street anomalies more challenging. In this work, an automatic color image analysis using a deep neural network for the detection of potholes on the road using images taken by a ZED camera is proposed. A lightweight architecture was designed to speed up training and usage. This consists of seven properly connected and synchronized layers. All the pixels of the original image are used without resizing. The classic stride and pooling operations were used to obtain as much information as possible. A database was built using a ZED camera seated on the front of a car. The routes where the photographs were taken are located in the city of Celaya in Guanajuato, Mexico. Seven hundred and fourteen images were manually tagged, several of which contain bumps and potholes. The system was trained with 70% of the database and validated with the remaining 30%. In addition, we propose a database that discriminates between potholes and speed bumps. A precision of 98.13% using 37 convolution filters in a 3 × 3 window was obtained, which improves upon recent state-of-the-art work. Full article

During the maneuvering of most unmanned aerial vehicles (UAVs), the GPS is one of the sensors used for navigation. However, this kind of sensor cannot handle indoor navigation applications well. Using a camera might be the answer to performing indoor navigation using its coordinate system. In this study, we considered indoor navigation applications using the ZED2 stereo camera for the quadcopter. To use the ZED 2 camera as a navigation sensor, we first transformed its coordinates into the North, East, down (NED) system to enable the drone to understand its position and maintain stability in a particular position. The experiment was performed using a real-time application to confirm the feasibility of this approach for indoor localization. In the real-time application, we commanded the quadcopter to follow triangular and rectangular paths. The results indicated that the quadcopter was able to follow the paths and maintain its stability in specific coordinate positions. Full article

This article discusses a model of a robotic platform that can be used for the proximal probing of biological objects in an ecologically balanced zone. The proximal probing is for scanning deciduous and fertile parts of biological objects with a hyperspectral camera at a distance of no more than a few meters. It allows for the obtention of information about the presence of phyto-diseases of tissues and also about the degree of ripeness and other parameters of the internal quality of the fruit. In this article, we report the methods and approaches used to detect fruits in the crown of a tree and also to identify their diseases such as scab and decay with an accuracy of at least 87%. For the autonomous movement of the platform in an ecologically balanced area, visual and inertial navigation is based on a Zed 2i stereo camera. This allows for the moving of biological objects in accordance with a given route indicated on the 2D map. The analysis of the information received from this platform allows for the building of maps of the presence of phyto-deseases in an ecologically balanced zone, and decisions are promptly made regarding the implementation of technical and protective measures that ensure high-quality products. Full article

Commercial visual–inertial odometry (VIO) systems have been gaining attention as cost-effective, off-the-shelf, six-degree-of-freedom (6-DoF) ego-motion-tracking sensors for estimating accurate and consistent camera pose data, in addition to their ability to operate without external localization from motion capture or global positioning systems. It is unclear from existing results, however, which commercial VIO platforms are the most stable, consistent, and accurate in terms of state estimation for indoor and outdoor robotic applications. We assessed four popular proprietary VIO systems (Apple ARKit, Google ARCore, Intel RealSense T265, and Stereolabs ZED 2) through a series of both indoor and outdoor experiments in which we showed their positioning stability, consistency, and accuracy. After evaluating four popular VIO sensors in challenging real-world indoor and outdoor scenarios, Apple ARKit showed the most stable and high accuracy/consistency, and the relative pose error was a drift error of about 0.02 m per second. We present our complete results as a benchmark comparison for the research community. Full article

In the traditional teleoperation system, the operator locates the object using the real-time scene information sent back from the robot terminal; however, the localization accuracy is poor and the execution efficiency is low. To address the issues, we propose an object detection and localization method for the teleoperated robot. First, we improved the classic YOLOv5 network model to produce superior object detection performance and named the improved model YOLOv5_Tel. On the basis of the classic YOLOv5 network model, the feature pyramid network was changed to a bidirectional feature pyramid network (BiFPN) network module to achieve the weighted feature fusion mechanism. The coordinate attention (CA) module was added to make the model pay more attention to the features of interest. Furthermore, we pruned the model from the depth and width to make it more lightweight and changed the bounding box regression loss function GIOU to SIOU to speed up model convergence. Then, the YOLOv5_Tel model and ZED2 depth camera were used to achieve object localization based on the binocular stereo vision ranging principle. Finally, we established an object detection platform for the teleoperated robot and created a small dataset to validate the proposed method. The experiment shows that compared with the classic YOLOv5 series network model, the YOLOv5_Tel is higher in accuracy, lighter in weight, and faster in detection speed. The mean average precision (mAP) value of the YOLOv5_Tel increased by 0.8%, 0.9%, and 1.0%, respectively. The model size decreased by 11.1%, 70.0%, and 86.4%, respectively. The inference time decreased by 9.1%, 42.9%, and 58.3%, respectively. The proposed object localization method has a high localization accuracy with an average relative error of only 1.12%. Full article

The growing share of the population over the age of 65 is putting pressure on the social health insurance system, especially on institutions that provide long-term care services for the elderly or to people who suffer from chronic diseases or mental disabilities. This pressure can be reduced through the assisted living of the patients, based on an intelligent system for monitoring vital signs and home automation. In this regard, since 2008, the European Commission has financed the development of medical products and services through the ambient assisted living (AAL) program—Ageing Well in the Digital World. The SmartCare Project, which integrates the proposed Computer Vision solution, follows the European strategy on AAL. This paper presents an indoor human activity recognition (HAR) system based on scene understanding. The system consists of a ZED 2 stereo camera and a NVIDIA Jetson AGX processing unit. The recognition of human activity is carried out in two stages: all humans and objects in the frame are detected using a neural network, then the results are fed to a second network for the detection of interactions between humans and objects. The activity score is determined based on the human–object interaction (HOI) detections. Full article