J. Imaging, Volume 8, Issue 12 (December 2022) – 20 articles

Single-photon emission computed tomography (SPECT) images can significantly help physicians in diagnosing patients with coronary artery or suspected coronary artery diseases. However, these images are grayscale with qualities that are not readily visible. The objective of this study was to evaluate the effectiveness of different pseudo-coloring algorithms of myocardial perfusion SPECT images. Data were collected using a Siemens Symbia T2 dual-head SPECT/computed tomography (CT) scanner. After pseudo-coloring, the images were assessed both qualitatively and quantitatively. The qualities of different pseudo-color images were examined by three experts, while the images were evaluated quantitatively by obtaining indices such as mean squared error (MSE), peak signal-to-noise ratio (PSNR), normalized color difference (NCD), and structure similarity index metric (SSIM). The qualitative evaluation demonstrated that the warm color map (WCM), followed by the jet color map, outperformed the remaining algorithms in terms of revealing the non-visible qualities of the images. Furthermore, the quantitative evaluation results demonstrated that the WCM had the highest PSNR and SSIM but the lowest MSE. Overall, the WCM could outperform the other color maps both qualitatively and quantitatively. The novelty of this study includes comparing different pseudo-coloring methods to improve the quality of myocardial perfusion SPECT images and utilizing our collected datasets. Full article

Objective: The application of computer models in continuous patient activity monitoring using video cameras is complicated by the capture of images of varying qualities due to poor lighting conditions and lower image resolutions. Insufficient literature has assessed the effects of image resolution, color depth, noise level, and low light on the inference of eye opening and closing and body landmarks from digital images. Method: This study systematically assessed the effects of varying image resolutions (from 100 × 100 pixels to 20 × 20 pixels at an interval of 10 pixels), lighting conditions (from 42 to 2 lux with an interval of 2 lux), color-depths (from 16.7 M colors to 8 M, 1 M, 512 K, 216 K, 64 K, 8 K, 1 K, 729, 512, 343, 216, 125, 64, 27, and 8 colors), and noise levels on the accuracy and model performance in eye dimension estimation and body keypoint localization using the Dlib library and OpenPose with images from the Closed Eyes in the Wild and the COCO datasets, as well as photographs of the face captured at different light intensities. Results: The model accuracy and rate of model failure remained acceptable at an image resolution of 60 × 60 pixels, a color depth of 343 colors, a light intensity of 14 lux, and a Gaussian noise level of 4% (i.e., 4% of pixels replaced by Gaussian noise). Conclusions: The Dlib and OpenPose models failed to detect eye dimensions and body keypoints only at low image resolutions, lighting conditions, and color depths. Clinical Impact: Our established baseline threshold values will be useful for future work in the application of computer vision in continuous patient monitoring. Full article

The margin of the removed tumor in cancer surgery has an important influence on survival. Adjuvant treatments, prognostic complications, and financial costs are required when the pathologist observes a close/positive surgical margin. Ex vivo imaging of resected cancer tissue has been suggested for margin assessment, but traditional cross-sectional imaging is not optimal in a surgical setting. Instead, three-dimensional (3D) ultrasound is a portable, high-resolution, and low-cost method to use in the operation room. In this study, we aimed to investigate the accuracy of 3D ultrasound versus computed tomography (CT) to measure the tumor volume in an animal model compared to gross pathology assessment. The specimen was formalin fixated before systematic slicing. A slice-by-slice area measurement was performed to compare the accuracy of the 3D ultrasound and CT techniques. The tumor volume measured by pathological assessment was 980.2 mm³. The measured volume using CT was 890.4 ± 90 mm³, and the volume using 3D ultrasound was 924.2 ± 96 mm³. The correlation coefficient for CT was 0.91 and that for 3D ultrasound was 0.96. Three-dimensional ultrasound is a feasible and accurate modality to measure the tumor volume in an animal model. The accuracy of tumor delineation on CT depends on the soft tissue contrast. Full article

Cross-Modal Hashing (CMH) retrieval methods have garnered increasing attention within the information retrieval research community due to their capability to deal with large amounts of data thanks to the computational efficiency of hash-based methods. To date, the focus of cross-modal hashing methods has been on training with paired data. Paired data refers to samples with one-to-one correspondence across modalities, e.g., image and text pairs where the text sample describes the image. However, real-world applications produce unpaired data that cannot be utilised by most current CMH methods during the training process. Models that can learn from unpaired data are crucial for real-world applications such as cross-modal neural information retrieval where paired data is limited or not available to train the model. This paper provides (1) an overview of the CMH methods when applied to unpaired datasets, (2) proposes a framework that enables pairwise-constrained CMH methods to train with unpaired samples, and (3) evaluates the performance of state-of-the-art CMH methods across different pairing scenarios. Full article

To train an automatic brain tumor segmentation model, a large amount of data is required. In this paper, we proposed a strategy to overcome the limited amount of clinically collected magnetic resonance image (MRI) data regarding meningiomas by pre-training a model using a larger public dataset of MRIs of gliomas and augmenting our meningioma training set with normal brain MRIs. Pre-operative MRIs of 91 meningioma patients (171 MRIs) and 10 non-meningioma patients (normal brains) were collected between 2016 and 2019. Three-dimensional (3D) U-Net was used as the base architecture. The model was pre-trained with BraTS 2019 data, then fine-tuned with our datasets consisting of 154 meningioma MRIs and 10 normal brain MRIs. To increase the utility of the normal brain MRIs, a novel balanced Dice loss (BDL) function was used instead of the conventional soft Dice loss function. The model performance was evaluated using the Dice scores across the remaining 17 meningioma MRIs. The segmentation performance of the model was sequentially improved via the pre-training and inclusion of normal brain images. The Dice scores improved from 0.72 to 0.76 when the model was pre-trained. The inclusion of normal brain MRIs to fine-tune the model improved the Dice score; it increased to 0.79. When employing BDL as the loss function, the Dice score reached 0.84. The proposed learning strategy for U-net showed potential for use in segmenting meningioma lesions. Full article

Advances in Artificial intelligence (AI) and embedded systems have resulted on a recent increase in use of image processing applications for smart cities’ safety. This enables a cost-adequate scale of automated video surveillance, increasing the data available and releasing human intervention. At the same time, although deep learning is a very intensive task in terms of computing resources, hardware and software improvements have emerged, allowing embedded systems to implement sophisticated machine learning algorithms at the edge. Additionally, new lightweight open-source middleware for constrained resource devices, such as EdgeX Foundry, have appeared to facilitate the collection and processing of data at sensor level, with communication capabilities to exchange data with a cloud enterprise application. The objective of this work is to show and describe the development of two Edge Smart Camera Systems for safety of Smart cities within S4AllCities H2020 project. Hence, the work presents hardware and software modules developed within the project, including a custom hardware platform specifically developed for the deployment of deep learning models based on the I.MX8 Plus from NXP, which considerably reduces processing and inference times; a custom Video Analytics Edge Computing (VAEC) system deployed on a commercial NVIDIA Jetson TX2 platform, which provides high level results on person detection processes; and an edge computing framework for the management of those two edge devices, namely Distributed Edge Computing framework, DECIoT. To verify the utility and functionality of the systems, extended experiments were performed. The results highlight their potential to provide enhanced situational awareness and demonstrate the suitability for edge machine vision applications for safety in smart cities. Full article

Tone mapping functions are applied to images to compress the dynamic range of an image, to make image details more conspicuous, and most importantly, to produce a pleasing reproduction. Contrast Limited Histogram Equalization (CLHE) is one of the simplest and most widely deployed tone mapping algorithms. CLHE works by iteratively refining an input histogram (to meet certain conditions) until convergence, then the cumulative histogram of the result is used to define the tone map that is used to enhance the image. This paper makes three contributions. First, we show that CLHE can be exactly formulated as a deep tone mapping neural network (which we call the TM-Net). The TM-Net has as many layers as there are refinements in CLHE (i.e., 60+ layers since CLHE can take up to 60 refinements to converge). Second, we show that we can train a fixed 2-layer TM-Net to compute CLHE, thereby making CLHE up to 30× faster to compute. Thirdly, we take a more complex tone-mapper (that uses quadratic programming) and show that it too can also be implemented — without loss of visual accuracy—using a bespoke trained 2-layer TM-Net. Experiments on a large corpus of 40,000+ images validate our methods. Full article

Automatic Speech Recognition (ASR) systems are ubiquitous in various commercial applications. These systems typically rely on machine learning techniques for transcribing voice commands into text for further processing. Despite their success in many applications, audio Adversarial Examples (AEs) have emerged as a major security threat to ASR systems. This is because audio AEs are able to fool ASR models into producing incorrect results. While researchers have investigated methods for defending against audio AEs, the intrinsic properties of AEs and benign audio are not well studied. The work in this paper shows that the machine learning decision boundary patterns around audio AEs and benign audio are fundamentally different. Using dimensionality-reduction techniques, this work shows that these different patterns can be visually distinguished in two-dimensional (2D) space. This in turn allows for the detection of audio AEs using anomal- detection methods. Full article

The application of chest X-ray imaging for early disease screening is attracting interest from the computer vision and deep learning community. To date, various deep learning models have been applied in X-ray image analysis. However, models perform inconsistently depending on the dataset. In this paper, we consider each individual model as a medical doctor. We then propose a doctor consultation-inspired method that fuses multiple models. In particular, we consider both early and late fusion mechanisms for consultation. The early fusion mechanism combines the deep learned features from multiple models, whereas the late fusion method combines the confidence scores of all individual models. Experiments on two X-ray imaging datasets demonstrate the superiority of the proposed method relative to baseline. The experimental results also show that early consultation consistently outperforms the late consultation mechanism in both benchmark datasets. In particular, the early doctor consultation-inspired model outperforms all individual models by a large margin, i.e., 3.03 and 1.86 in terms of accuracy in the UIT COVID-19 and chest X-ray datasets, respectively. Full article

One way to diagnose a disease is to examine pictures of tissue thought to be affected by the disease. Near-infrared properties are subdivided into nonionizing, noninvasive, and nonradiative properties. Near-infrared also has selectivity properties for the objects it passes through. With this selectivity, the resulting attenuation coefficient value will differ depending on the type of material or wavelength. By measuring the output and input intensity values, as well as the attenuation coefficient, the thickness of a material can be measured. The thickness value can then be used to display a reconstructed image. In this study, the object studied was a phantom consisting of silicon rubber, margarine, and gelatin. The results showed that margarine materials could be decomposed from other ingredients with a wavelength of 980 nm. Full article

Glioblastoma Multiforme (GBM) is considered one of the most aggressive malignant tumors, characterized by a tremendously low survival rate. Despite alkylating chemotherapy being typically adopted to fight this tumor, it is known that O(6)-methylguanine-DNA methyltransferase (MGMT) enzyme repair abilities can antagonize the cytotoxic effects of alkylating agents, strongly limiting tumor cell destruction. However, it has been observed that MGMT promoter regions may be subject to methylation, a biological process preventing MGMT enzymes from removing the alkyl agents. As a consequence, the presence of the methylation process in GBM patients can be considered a predictive biomarker of response to therapy and a prognosis factor. Unfortunately, identifying signs of methylation is a non-trivial matter, often requiring expensive, time-consuming, and invasive procedures. In this work, we propose to face MGMT promoter methylation identification analyzing Magnetic Resonance Imaging (MRI) data using a Deep Learning (DL) based approach. In particular, we propose a Convolutional Neural Network (CNN) operating on suspicious regions on the FLAIR series, pre-selected through an unsupervised Knowledge-Based filter leveraging both FLAIR and T1-weighted series. The experiments, run on two different publicly available datasets, show that the proposed approach can obtain results comparable to (and in some cases better than) the considered competitor approach while consisting of less than $0.29\%$ of its parameters. Finally, we perform an eXplainable AI (XAI) analysis to take a little step further toward the clinical usability of a DL-based approach for MGMT promoter detection in brain MRI. Full article

Self-supervised learning approaches have seen success transferring between similar medical imaging datasets, however there has been no large scale attempt to compare the transferability of self-supervised models against each other on medical images. In this study, we compare the generalisability of seven self-supervised models, two of which were trained in-domain, against supervised baselines across eight different medical datasets. We find that ImageNet pretrained self-supervised models are more generalisable than their supervised counterparts, scoring up to 10% better on medical classification tasks. The two in-domain pretrained models outperformed other models by over 20% on in-domain tasks, however they suffered significant loss of accuracy on all other tasks. Our investigation of the feature representations suggests that this trend may be due to the models learning to focus too heavily on specific areas. Full article

This work aims to leverage medical augmented reality (AR) technology to counter the shortage of medical experts in low-resource environments. We present a complete and cross-platform proof-of-concept AR system that enables remote users to teach and train medical procedures without expensive medical equipment or external sensors. By seeing the 3D viewpoint and head movements of the teacher, the student can follow the teacher’s actions on the real patient. Alternatively, it is possible to stream the 3D view of the patient from the student to the teacher, allowing the teacher to guide the student during the remote session. A pilot study of our system shows that it is easy to transfer detailed instructions through this remote teaching system and that the interface is easily accessible and intuitive for users. We provide a performant pipeline that synchronizes, compresses, and streams sensor data through parallel efficiency. Full article

In this paper, we propose an advanced scripting approach using Python and R for satellite image processing and modelling terrain in Côte d’Ivoire, West Africa. Data include Landsat 9 OLI/TIRS C2 L1 and the SRTM digital elevation model (DEM). The EarthPy library of Python and ‘raster’ and ‘terra’ packages of R are used as tools for data processing. The methodology includes computing vegetation indices to derive information on vegetation coverage and terrain modelling. Four vegetation indices were computed and visualised using R: the Normalized Difference Vegetation Index (NDVI), Enhanced Vegetation Index 2 (EVI2), Soil-Adjusted Vegetation Index (SAVI) and Atmospherically Resistant Vegetation Index 2 (ARVI2). The SAVI index is demonstrated to be more suitable and better adjusted to the vegetation analysis, which is beneficial for agricultural monitoring in Côte d’Ivoire. The terrain analysis is performed using Python and includes slope, aspect, hillshade and relief modelling with changed parameters for the sun azimuth and angle. The vegetation pattern in Côte d’Ivoire is heterogeneous, which reflects the complexity of the terrain structure. Therefore, the terrain and vegetation data modelling is aimed at the analysis of the relationship between the regional topography and environmental setting in the study area. The upscaled mapping is performed as regional environmental analysis of the Yamoussoukro surroundings and local topographic modelling of the Kossou Lake. The algorithms of the data processing include image resampling, band composition, statistical analysis and map algebra used for calculation of the vegetation indices in Côte d’Ivoire. This study demonstrates the effective application of the advanced programming algorithms in Python and R for satellite image processing. Full article

Accurate and rapid crop type mapping is critical for agricultural sustainability. The growing trend of cloud-based geospatial platforms provides rapid processing tools and cloud storage for remote sensing data. In particular, a variety of remote sensing applications have made use of publicly accessible data from the Sentinel missions of the European Space Agency (ESA). However, few studies have employed these data to evaluate the effectiveness of Sentinel-1, and Sentinel-2 spectral bands and Machine Learning (ML) techniques in challenging highly heterogeneous and fragmented agricultural landscapes using the Google Earth Engine (GEE) cloud computing platform. This work aims to map, accurately and early, the crop types in a highly heterogeneous and fragmented agricultural region of the Tadla Irrigated Perimeter (TIP) as a case study using the high spatiotemporal resolution of Sentinel-1, Sentinel-2, and a Random Forest (RF) classifier implemented on GEE. More specifically, five experiments were performed to assess the optical band reflectance values, vegetation indices, and SAR backscattering coefficients on the accuracy of crop classification. Besides, two scenarios were used to assess the monthly temporal windows on classification accuracy. The findings of this study show that the fusion of Sentinel-1 and Sentinel-2 data can accurately produce the early crop mapping of the studied area with an Overall Accuracy (OA) reaching 95.02%. The scenarios prove that the monthly time series perform better in terms of classification accuracy than single monthly windows images. Red-edge and shortwave infrared bands can improve the accuracy of crop classification by 1.72% when compared to only using traditional bands (i.e., visible and near-infrared bands). The inclusion of two common vegetation indices (The Normalized Vegetation Index (NDVI), the Enhanced Vegetation Index (EVI)) and Sentinel-1 backscattering coefficients to the crop classification enhanced the overall classification accuracy by 0.02% and 2.94%, respectively, compared to using the Sentinel-2 reflectance bands alone. The monthly windows analysis indicated that the improvement in the accuracy of crop classification is the greatest when the March images are accessible, with an OA higher than 80%. Full article

Some color images primarily comprise specific hues, for example, food images predominantly contain a warm hue. Therefore, these hues are essential for creating delicious impressions of food images. This paper proposes a color image enhancement method that can select hues to be enhanced arbitrarily. The current chroma is considered such that near achromatic colors are not over-enhanced. The effectiveness of the proposed method was confirmed through experiments using several food images. Full article

Tissue characterization by mapping techniques is a recent magnetic resonance imaging (MRI) tool that could aid the tissue characterization of lung parenchyma in coronavirus disease-2019 (COVID-19). The aim of the present study was to compare lung MRI findings, including T1 and T2 mapping, in a group of n = 11 patients with COVID-19 pneumonia who underwent a scheduled cardiac MRI, and a cohort of healthy controls. MRI scout images were used to identify affected and remote lung regions within the patients’ cohort and appropriate regions of interest (ROIs) were drawn accordingly. Both lung native T1 and T2 values were significantly higher in the affected areas of patients with COVID-19 as compared to the controls (1375 ms vs. 1201 ms, p = 0.016 and 70 ms vs. 30 ms, p < 0.001, respectively), whereas no significant differences were detected between the remote lung parenchyma of the COVID-19 patients and the controls (both p > 0.05). When a larger ROI was identified, comprising the whole lung parenchyma within the image irrespective of the affected and remote areas, the COVID-19 patients still retained higher native T1 (1278 ms vs. 1149 ms, p = 0.003) and T2 values (38 ms vs. 34 ms, p = 0.04). According to the receiver operator characteristics curves, the T2 value of the affected region retained the higher accuracy for the differentiation of the COVID-19 patients against the controls (area under the curve 0.934, 95% confidence interval 0.826–0.999). These findings, possibly driven by the ability of MRI tissue mapping to detect ongoing inflammation in the lungs of patients with COVID-19, suggest that T1 and T2 mapping of the lung is a feasible approach in this clinical scenario. Full article

The location of the macular central is very important for the examination of macular edema when using an automated screening system. The erratic character of the macular light intensity and the absence of a clear border make this anatomical structure difficult to detect. This paper presents a new method for detecting the macular center based on its geometrical location in the temporal direction of the optic disc. Also, a new method of determining the temporal direction using the vascular features visible on the optic disc is proposed. After detecting the optic disc, the temporal direction is determined by considering blood vessel positions. The macular center is detected using thresholding and simple morphology operations with optimum macular region of interest (ROI) direction. The results show that the proposed method has a low computation time of 0.34 s/image with 100% accuracy for the DRIVE dataset, while that of DiaretDB1 was 0.57 s/image with 98.87% accuracy. Full article

Seismic full wave inversion (FWI) is a widely used non-linear seismic imaging method used to reconstruct subsurface velocity images, however it is time consuming, has high computational cost and depend heavily on human interaction. Recently, deep learning has accelerated it’s use in several data-driven techniques, however most deep learning techniques suffer from overfitting and stability issues. In this work, we propose an edge computing-based data-driven inversion technique based on supervised deep convolutional neural network to accurately reconstruct the subsurface velocities. Deep learning based data-driven technique depends mostly on bulk data training. In this work, we train our deep convolutional neural network (DCN) (UNet and InversionNet) on the raw seismic data and their corresponding velocity models during the training phase to learn the non-linear mapping between the seismic data and velocity models. The trained network is then used to estimate the velocity models from new input seismic data during the prediction phase. The prediction phase is performed on a resource-constrained edge device such as Raspberry Pi. Raspberry Pi provides real-time and on-device computational power to execute the inference process. In addition, we demonstrate robustness of our models to perform inversion in the presence on noise by performing both noise-aware and no-noise training and feeding the resulting trained models with noise at different signal-to-noise (SNR) ratio values. We make great efforts to achieve very feasible inference times on the Raspberry Pi for both models. Specifically, the inference times per prediction for UNet and InversionNet models on Raspberry Pi were 22 and 4 s respectively whilst inference times for both models on the GPU were 2 and 18 s which are very comparable. Finally, we have designed a user-friendly interactive graphical user interface (GUI) to automate the model execution and inversion process on the Raspberry Pi. Full article