Applications of Computer Vision, 2nd Edition

1. Introduction to the Applications of Computer Vision

Computer vision (CV) is a broad term mainly used to refer to processing image and video data. CV aims to enable machines to perceive, observe, and understand the physical world as if they have human eyes. While this area of knowledge began to develop during the 1970s and 1980s, the last three decades have been characterized by the field maturing. This progress can be seen in the increasing number of software and hardware products on the market, the significant growth of active applications, and the rise in recent scientific publications on this research area. The first applications of computer vision were in the field of medical imaging and the processing of remote sensing data. Hence, the scientific journals IEEE Transaction on Medical Imaging and IEEE Transactions on Geoscience and Remote Sensing were created by the Institute of Electrical and Electronics Engineers (IEEE) association in 1982 and 1980, respectively, to manage the engineering aspects for medical imaging and satellite data.

Remote sensing (RS) images are obtained using remote sensing technology such as airplanes and satellites under long-distance conditions. The detection of targets in RS images is very important in military applications, urban planning, resource exploration, agriculture, and other fields. CV techniques like content-based image retrieval techniques [1], semantic segmentation [2], scene classification [3], nonsupervised learning [4], and transfer learning [4], among others, have been applied to RS images. One specific application is cropland field identification, which is a key element of precision agriculture [5].

Common imaging techniques like X-ray radiographs, computed tomography (CT), and/or magnetic resonance imaging (MRI) have revolutionized the field of diagnostic medicine, providing non-destructive procedures for examining the interior of our bodies. Due to overlaps between anatomical structures, interpreting medical images is very challenging, even for experienced radiologists. The clinical interest in understanding these medical images explains the interest in developing computer algorithms that can aid experts in their clinical tasks. Across 40 years, the intersection of CV techniques and medical imaging has provided many clinical solutions. Common CV tasks like feature detection, recognition, segmentation, and three-dimensional modeling have been developed for processing different types of medical images and solving specific clinical problems. Some examples are chest radiograph analysis [6]; dental imaging (panoramic X-rays and other imaging modalities), to aid dental experts in diagnosing various dental disorders [7]; brain MRI modalities, for identifying distinct features that characterize autism spectrum disorder [8]; skin lesion analysis from RGB images to diagnosis skin cancer [9]; diagnosing glaucoma by analyzing retinal imaging data [10]; and detecting lung and colorectal cancer using CT imaging [11,12]. Recent advances in robotics now permit the acquisition of more medical images that can help clinicians make diagnoses or guide surgeons, in which the source and detector are positioned by robots with greater precision and accuracy [13,14]. Although X-ray imaging technology has been used in clinical tasks for decades, it has recently been extended to industrial production and security applications, where it can detect anomalies or defects inside products non-destructively and identify prohibited objects inside baggage without opening it [15].

Machine vision systems make use of different processing stages like image pre-processing, target image or video segmentation, feature extraction and selection, object recognition, classification, and 3D modeling, among others. These different types of tasks have typically involved different types of algorithms, with the classical CV techniques using explicitly programmed algorithms to solve specific tasks [16,17]. In recent years, deep learning (DL) models have yielded a new generation of CV methods [18] based on multi-layered neural networks such as convolutional neural networks and transformers, endowing computers with the ability to learn without them being explicitly programmed. The most popular architectures for computer vision are convolutional neural networks (CNNs) [19], which have become the standard DL-based approches for many recognition tasks due to their ability to learn high-level features in their convolutional layers; generative adversarial networks (GANs) [20], which learn from a given training set to generate new data; recurrent neural networks (RNNs), which have the capability to process temporal information and sequential data; different versions of YOLO (You Only Look Once) for object detection [21,22]; and transformers [23], which are primarily based on self-attention mechanisms [24]. These have all found applications in numerous fields, such as medicine [8,25,26,27,28], image generation [29], and remote sensing [2,4], among others. In conclusion, several algorithms have emerged over time, each with its own set of advantages and disadvantages. While DL models have good learnability, they often require a substantial number of real labels for training, provide poor interpretability due to their black-box structure, and require intensive computational resources or specific hardware.

As previously mentioned, the medical and remote sensing fields have used CV techniques for the automation of different tasks extensively. Nevertheless, recent advances in the image acquisition technology available, mainly due to research in optics and digital sensing, as well as increasing computer power, have unleashed new opportunities to apply CV techniques to new types of images, like microscopy imaging [30] or unmanned aerial vehicle (UAV) acquisitions [31]. UAVs are flying robots either remotely controlled by somebody or navigated autonomously using a computer system on board the vehicle or on the ground. They are able to acquire images in complex applications due to their small size, low cost, and high mobility. UAV systems enable the acquisition of real-time environmental data for developing CV applications such as vehicle detection [32] and digital precision agriculture [33,34,35], with the latter involving a variety of tasks, such as weed, crop pest, and disease detection, in order to apply the right practice at the right place, the right time, and the right quantity. Thus, UAVs are versatile, with the capacity for different kinds of sensors to be boarded onto them [36]. In capturing both the spatial and spectral features of an object’s surface, hyperspectral images are also used for agricultural tasks like disease detection, weed, and stress detection; crop monitoring; applying nutrients; soil mineralogy studies; yield estimation; and sorting applications [37,38].

Microscopy imaging has a prominent role in modern biology for the visualization of tissues, cells, proteins, and macromolecular structures at all resolutions. Indeed, biopsy diagnosis is the gold standard for cancer diagnosis in pathology. Machine vision has recently been employed in the biomedical field to detect, measure, and recognize cells and patterns in histopathology images or for target tracking and 3D reconstruction [28,39]. These biomedical applications can be grouped together on the basis of the tissue or organ analyzed—for example, renal pathology [40,41], computational cytology [42], breast cancer [43], oral cancer [44], and intestine pathology [45], among others. However, microscopy imaging has also found applications to pollen identification [46,47], microorganism recognition [48], and estimating the fecundity of fish based on histological images of their gonads [49,50].

CV techniques have also played an important role in the product life cycle across the entire industrial manufacturing process, including product design, modeling and simulation, planning and scheduling, the production process, inspection and quality control, assembly, transportation, and disassembly [51,52]. Equally, they have been applied to a myriad of domains: car parking lot management, detecting the positions of parking spaces [53] or used in autonomous driving [54]; the mushroom industry, for the identification of poisonous mushrooms, plucking cultivated mushrooms covered by the soil, and mechanized grading of mushrooms [55]; continuous monitoring of beehives [56] and beehive products such as honeybee pollen [57]; marine ecosystems, in monitoring fish habitats using underwater videos or images [58] or estimating the fecundity of fish from histological images of their gonads [50,59]; crop disease monitoring [35,60,61]; the identification of insects from digital images [62]; food quality assessments [63,64], covering potatoes [65], fruit damage [66], and dry-cured ham [67,68]; automation within the chicken farming industry [69]; and plant identification [70].

All of these computer vision applications involve the integration of the following elements:

• Support for data recording: Microscopes; UAVs; satellites; robots; MRI, X-ray, and CT devices; and others.
• Type of input data: 2D images, videos, or other information, dependent on the high-performance sensors used to perceive the given scenario, which could be RGB cameras; multispectral, hyperspectral, thermal, and infrared sensors; synthetic-aperture radar (SAR) cameras; Light Detection and Ranging or Laser Imaging Detection and Ranging (LiDAR) sensors; or other cameras [71].
• Machine vision-related aim of the application: Feature detection or recognition, image segmentation, image classification, 3D modeling or reconstruction, object tracking, defect detection, object counting or measurements from images, and visual inspection, among others. The evaluation methodology used in CV techniques is dependent on the aims and application in question.
• Type of processing: CV methods can be roughly divided into three categories: non-learning-based methods, learning-based methods, and hybrid methods. The first types of methods are usually known as the classical methods, and these rely on unsupervised, manually designed feature extractors or statistical models, in which the output is calculated from direct processing of the input data. Currently, the second types are methods based on deep learning, in which previous training with ground-truth data is needed to compute the output. Hybrid strategies normally combine the extraction of features from the input data with a subsequent machine learning stage.
• Experimental testing: Using publicly available datasets or private data.

Despite the abundance of works and reviews published in this domain in recent years, many challenges are still open questions. From a computational point of view, future work should focus on designing more efficient algorithms that can operate in real time or run on low-capacity devices such as UAVs. As mentioned, some machine vision techniques require a substantial amount of ground-truth labeled data for training. Transfer learning or unsupervised annotation algorithms have been proposed to alleviate the need for labeled data, addressing domain shift or directly labeling the data, but there is still room for further research on this aspect. At the same time, a considerable number of the studies in the literature on CV only use private data or use public datasets with different experimental setups, complicating comparison between algorithms. So, new data must be made public in order for the field to mature.

For CV applications in which the decision-making involved affects people, there is substantial evidence that AI-based systems take on race-, ethnicity-, culture-, age-, and gender-based biases, among others, that disadvantage minority populations. Gender bias typically intersects with other biases [72], and Natural Language Processing (NLP) and facial analysis and recognition are research fields that feel greater effects of gender bias—for example, gender bias in commercial facial recognition systems [73,74] or the social impact of image generation models [75]. Machine vision systems perpetuated or intensified social inequalities in recent applications of developing systems that integrated NLP and CV [76,77], with biases introduced by both. Biases can be introduced into CV systems in many different ways: the ground-truth labeling of the data, the selection of the data included for training, the algorithm design, and evaluation of the prediction quality, among other design decisions. Gender biases can be imbued into CV systems unintentionally due to our cultural experience or gender stereotypes. Therefore, CV system developers should be aware of gender bias in their future work. Equally, some of the images and videos used in CV research are obtained without the explicit consent of the people photographed. Hence, recently, the IEEE announced that it will no longer allow the use of the Lena image in its publications. Furthermore, in some CV applications, such as emotional computing, people’s right to privacy and intimacy should be socially debated [78].

2. Overview of This Special Issue

This Special Issue called for scientific articles related to the computer vision applications previously covered, and after a double-blind review process, nineteen articles were published. This section provides a brief overview of each contribution in order to encourage further exploration on the part of the reader.

The first contribution, entitled “A UAV Aerial Image Target Detection Algorithm Based on YOLOv7 Improved Model”, proposes an enhanced YOLOv7 model for detecting small targets in UAV images. Experiments were carried out on the UAV aerial photo dataset VisDrone2019 and compared with the YOLOv7 model.

The second contribution, entitled “RN-YOLO: A Small Target Detection Model for Aerial Remote-Sensing Images”, applies a new YOLO model based on YOLOv8, called RN-YOLO, to detecting small targets in RS images. These experiments were conducted on the TGRS-HRRSD and RSOD datasets and compared with the YOLOv8 model.

The third contribution, entitled “Dense Object Detection Based on De-Homogenized Queries”, establishes a new method for dense object detection in images and videos. Experiments were run on the CrowdHuman dataset and compared with other state-of-the-art (SOTA) methods.

The fourth contribution, entitled “Multi-Scale Fusion Uncrewed Aerial Vehicle Detection Based on RT-DETR”, covers an enhanced model of a real-time detection transformer (RT-DETR), a real-time end-to-end object detection model for detecting drones in images. Two available UAV datasets were used for the experiments.

The fifth contribution, entitled “Efficient Vision Transformer YOLOv5 for Accurate and Fast Traffic Sign Detection”, details a new model for detecting traffic signs, which is a vital task in autonomous driving systems. It achieved faster and more accurate results than the YOLOv5 model. Experiments were conducted on the 3L-TT100K traffic sign dataset.

The sixth contribution, entitled “Facial Beauty Prediction Combined with Multi-Task Learning of Adaptive Sharing Policy and Attentional Feature Fusion”, presents a strategy for improving facial attractiveness assessments, involving experimental testing on the LSAFBD and SCUT-FBP5500 databases.

The seventh contribution, entitled “Two-Stage Progressive Learning for Vehicle Re-Identification in Variable Illumination Condition”, elucidates a TSPL framework for recognizing vehicles in images acquired by surveillance cameras with varying viewpoints, levels of illumination, and resolutions. A private large-scale dataset (VERI-DAN) and the Vehicle-1M dataset were used for the experiments, and the framework proposed was compared with other SOTA methods.

Inspired by the separation of luminance and chrominance information in the YCbCr color space, the eighth contribution, entitled “DBENet: Dual-Branch Brightness Enhancement Fusion Network for Low-Light Image Enhancement”, describes a new model for enhancing RGB images with low light, minimizing brightness, color distortion, and noise pollution in the enhanced images. The experiments in this paper made use of multiple publicly available low-light image datasets, and the results were evaluated against those of classical algorithms.

The ninth contribution, entitled “RSLC-Deeplab: A Ground Object Classification Method for High-Resolution Remote Sensing Images”, suggests a semantic segmentation network for accurately segmenting remote sensing images. Experiments conducted using the WHDLD dataset demonstrated its outperformance of the PSP-NET, U-NET, MACU-NET, and DeeplabV3+ networks.

The tenth contribution, entitled “YOLO-CID: Improved YOLOv7 for X-ray Contraband Image Detection”, augments the YOLOv7 method for contraband image detection in X-ray inspection systems in order to detect small objects under occlusion or low contrast. Its results on the PIDray public dataset were an improvement upon the results of the YOLOv7 algorithm.

The eleventh contribution, entitled “Enhancing the Accuracy of an Image Classification Model Using Cross-Modality Transfer Learning”, proposes a cross-modality transfer learning approach to shifting the knowledge when the source and target domains are different, specifically from the text domain to the image domain.

The twelfth contribution, entitled “Three-Dimensional Measurement of Full Profile of Steel Rail Cross-Section Based on Line-Structured Light”, solves the industrial problem of improving railway operation safety by proposing a method for three-dimensional measurement of the cross-sectional profiles of steel rails based on binocular line-structured light. Private data were used in this paper.

The thirteenth contribution, entitled “A Workpiece-Dense Scene Object Detection Method Based on Improved YOLOv5”, optimizes the YOLOv5 method for detecting workpieces in dense images of industrial production lines, using a self-built artifact dataset to compare the results with the original method.

The fourteenth contribution, entitled “Improving the Performance of the Single Shot Multibox Detector for Steel Surface Defects with Context Fusion and Feature Refinement”, devises a method for improving the ability to identify steel surface defects. Experiments were run on the public NEU-DET dataset and compared with other SOTA methods (Faster R-CNN, RetinaNet, and different YOLO methods).

The fifteenth contribution, entitled “Object Detection Algorithm of UAV Aerial Photography Image Based on Anchor-Free Algorithms”, constructs an algorithm for anchor-free target detection in UAV aerial photography images. In experiments performed on the VisDrone dataset, it outperformed the fully convolutional one-stage object detection algorithm.

The sixteenth contribution, entitled “A Vehicle Recognition Model Based on Improved YOLOv5”, in order to increase vehicle driving safety, validated the results of an improved YOLOv5s algorithm for vehicle identification and detection on a self-built dataset against the results of the YOLOv5 method.

The seventeenth contribution, entitled “Material-Aware Path Aggregation Network and Shape Decoupled SIoU for X-ray Contraband Detection”, outlines a method, based on variants of the YOLO model, for detecting and classifying contraband in X-ray baggage images. They evaluated the method proposed on the public X-ray contraband SIXray and OPIXray datasets and conducted a comparison of the results with those of other SOTA X-ray baggage inspection detection methods.

The eighteenth contribution, entitled “Fast Adaptive Binarization of QR Code Images for Automatic Sorting in Logistics Systems”, presents an adaptive binarization method for reading unevenly illuminated QR codes in automatic sorting in logistics systems. The image quality, recognition rate, and computation speed of the proposed method was tested against other SOTA methods on different examples.

The nineteenth contribution, entitled “Surveying Racial Bias in Facial Recognition: Balancing Datasets and Algorithmic Enhancements”, is a review on facial recognition systems that involve specific racial categories, discussing balanced facial recognition datasets, addressing and analyzing the racial bias of the methods, and exploring the interrelation of racial and gender bias.