A Comprehensive Review on Unsupervised Domain Adaptation for 3D Segmentation and Reconstruction in CT Urography Imaging ^†

Abstract

Computed tomography urography (CTU) is a specialized radiological procedure that produces finely detailed pictures of the urinary system, comprising the kidneys, ureters, and bladder, using computed tomography (CT) scans. This diagnostic procedure’s main goal is to assess disorders that impact these vital organs, such as stones in the kidneys, tumors, UTIs, and morphological anomalies. CTU has benefits like the capacity to deliver a personalized therapeutic strategy via radiomics and artificial intelligence technologies, as well as extra knowledge about abdominal anatomy. This comprehensive article looks at how computed tomography urography (CTU) is used and how it can be changed to evaluate the urinary system, especially the kidneys, bladder, and ureters. The most important part of this review is the discussion on 3D kidney segmentation and reconstruction from urographic images, which has helped doctors a lot with the accurate diagnosis and planning of treatment for kidney diseases. Even though 3D convolution networks have been used a lot in medical picture segmentation, it can be hard to adapt them to clinical data from different modalities that have not been seen before. The review gives an in-depth look at the current research on how an unsupervised domain adaptation or translation method can be used with 2D networks, especially for accurate kidney segmentation in urographic images. Through this thorough study, we want to show how these techniques can be used in medical imaging and how they might change in the future.

1. Introduction

In retrograde intrarenal surgery (RIRS), a form of minimally invasive surgical treatment, kidney stones are removed using a flexible ureteroscope. The success rate and stone-free rate of RIRS depend on the surgical design of different anatomical three-dimensional kidney shape features, such as the infundibulopelvic tendons, infundibular size, and infundibular width [1]. RIRS has advantages such as decreased stress, faster recovery, and fewer complications. Before starting surgical planning for RIRS, these parameters must be carefully recognized in 3D kidney segmentation. Techniques for segmenting the kidneys in numerous ways have been recorded. Deep-learning-based kidney-harvesting techniques are presently the subject of in-depth investigation. In many renal disorders, kidney volume is severely impacted. It will be easier to size the kidney and value renal functions with precise and automatic segmentation. A crucial component of creating any non-invasive computer-aided diagnostic (CAD) system for kidney disorder early detection using urological pictures is kidney segmentation. To help doctors diagnose kidney and bladder stone issues, CT programs utilize imaging and contrast dye. Fully convolutional neural networks have proved quite effective at segmenting medical photographs, but they become stuck in unknown clinical data and cannot be changed in various modes with only one training method. To learn about urographic images for exact kidney segmentation, this paper explores the possibilities of different dimensional networks with unsupervised domain adaptation. Unsupervised domain translation, which tries to create a mapping that can transport images from an original domain to a target domain using unpaired training data, has attracted a lot of attention in recent years [2]. It is possible to create a tagged synthetic training set using a generative antagonistic network [3] in cases where authentic labels for actual medical images are unavailable. Articles [4,5] give precise CTU kidney separation utilizing an unsupervised domains translation technique and recommend employing two-dimensional deep learning convolutional networks.

The suggested method automatically creates the kidney CTU-like images and transforms kidney CT scans with trained neural network models into the CTU domain in order to segment the structural CTU kidneys, eliminating the requirement for manually labeled CTU images. The verification of the clinical renal CTU data shows that our proposed method for segmenting the kidney outperforms the supervised learning-based alternatives. There are two segmentation and production steps in the approach. Cycle-consistent adversarial networks with generative features [6] are trained to generate labeled CTU-like pictures in the generation phase with labeled CT as well as unlabeled CTU data. A two-dimensional U-Net-like network has been trained utilizing the CTU-like data in the segmentation step. Both sessions provide end-to-end training. Our suggested approach may effectively adjust the data shift and create accurate kidney segmentation using CTU images without the need for human data annotation.

With a goal to investigate an unsupervised domain adaptation method based on adversarial learning [7], a study by Huo and team deliberated on properly segregating the kidneys on CTU images without utilizing labeled CTU data for training deep convolution neural networks. It is specifically used for surgical planning and outcome analysis prior to and following renal surgery. Doctors may accurately diagnose and treat kidney problems using an intuitive visualization method called kidney three-dimensional segmentation and reconstructions from urographic images [8].

The Clinical Significance

Clinically meaningful 3D kidney segmentation and reconstruction in CTU is necessary since it directly affects patient care and medical diagnosis. For lesion identification and treatment planning, a precise division of kidney tumors using CT volumes is crucial [9]. For the purpose of keeping track of renal illnesses, accurate and automatic segmentation of the kidneys can be used to measure kidney size and assess renal function [10]. Additionally, the three-dimensional reconstruction of the kidney CT picture using the segmentation results offers a simple and reliable diagnostic tool [11]. The ability to quantitatively characterize these lesions through semantic segmentation of kidney tumors and their host kidneys improves perioperative and oncologic outcomes [12]. Furthermore, the development of non-invasive computer-assisted diagnostic methods for assessing renal function is made possible by 3D kidney segmentation from dynamic CT images, which enhances patient care [13]. In general, 3D kidney segmentation and reconstruction in CTU are essential for clinical diagnosis, treatment planning, and patient monitoring of renal disorders, all of which result in better patient outcomes.

2. Methodology

A key component of computer-aided diagnosis and therapy planning for kidney illnesses is renal segmentation in medical imaging. Researchers have proposed a variety of methods for precisely segmenting kidneys and kidney cancers in light of improvements in imaging acquisition technologies like computed tomography urography (CTU) and the use of magnetic resonance imaging (MRI), as well as Ultrasound. In this paper, we present a methodical examination of the techniques created particularly for kidney segmentation in the CTU imaging. With a focus on their theoretical underpinnings and performance indicators, the paper tries to classify and compare several segmentation techniques. This study aims to answer important research concerns about cutting-edge kidney segmentation methods and their therapeutic consequences by combining material from the literature. We hope that our project will shed useful light on the subject, point out areas that could use development, and highlight the importance of accurate kidney segmentation in medical practice.

2.1. Objective of the Review

This review’s goal is to offer a thorough evaluation of the techniques created for kidney segmentation in CTU scanning. The review’s objectives are to provide a summary of the various segmentation algorithms, classify them according to their theoretical underpinnings, and assess how well they function when effectively segmenting kidney and kidney cancers.

2.2. Research Questions

The review paper examines the following research questions to fulfill the previously stated goal:

RQ1

What are the key methodologies developed for kidney segmentation in CTU imaging?

RQ2

How do different segmentation algorithms perform in terms of accuracy and computational efficiency?

RQ3

What are the applications and clinical significance of accurate kidney segmentation in CTU imaging?

2.3. Data Collection and Analysis

The review article’s scope is specifically defined to provide an in-depth look at recent developments in the field. Peer-reviewed papers and articles from 2014 to 2021 that emphasize the clinical uses and implications of unsupervised domain adaption approaches, 3D segmentation and reconstruction procedures, and CT urography imaging are included in the review. The scope excludes supervised learning strategies, unreviewed papers, unrelated medical imaging methods, and specific case studies with scant scientific support. Using keywords related to 3D segmentation and CT urography imaging, a thorough search was conducted in databases like PubMed, IEEE Xplore, and Springer.

2.4. Data Synthesis

To provide a thorough picture of kidney segmentation in CTU imaging, the data were synthesized, combining the results from various research and approaches. The evaluation includes a summary of the main techniques, together with information about their applications, limits, and performance indicators.

3. Background

3.1. Adversarial Networks

A novel method for generative modeling is called generative adversarial networks (GANs). A generator model for producing new instances and a discriminator model for classifying examples as real or produced make up the GANs’ framework, which converts generative modeling into a supervised training issue. The effectiveness of the generator is determined by training the models collectively until the discriminator is misled around half the time. Notably, GANs have achieved great success in jobs requiring image-to-image translation and in creating photorealistic images that frequently deceive people.

3.2. Unsupervised Domain Translation

Unsupervised domain translation simply requires an independent training sample from every domain and does not require paired data. CycleGAN, a GAN extension that excels in picture translation tasks, is a noteworthy example of such an application. By matching the training samples within a cycle consistency constraint, it finds the mappings between domains. This strategy has also proven effective in Natural Language Processing (NLP), where techniques like cross-lingual language models and dual learning produce excellent language translation results. Unsupervised domain translation techniques can fall short on a sizable portion of test samples, notwithstanding their accomplishments.

The unsupervised domain adaptation workflow for CTU imaging, depicted in Figure 1, outlines the various stages of the process.

In the context of medical imaging, Figure 2 provides an overview of the generic workflow encompassing CT scanning and the subsequent 3D kidney segmentation process.

4. Results

In recent years, there has been a notable increase in the use of unsupervised domain adaption approaches for three-dimensional segmentation and reconstruction in CT urography imaging. Our thorough analysis covers the range of techniques and their varied approaches in this field.

Several studies showed great levels of segmentation accuracy, as shown by their individual Jaccard indices and Dice Similarity Coefficient values. For instance, the multi-atlas image registration method produced a Dice Similarity Coefficient of 0.952 [14], whereas the MSS U-Net technique, which is more sophisticated, achieved a Dice Similarity Coefficient of 0.969 [15]. This range of outcomes shows that, despite the use of numerous approaches, many people have achieved an admirable accomplishment in their attempts.

The applications of these strategies in the real world must also be emphasized. For example, techniques like the one described in [8] do not just support the diagnosis of renal illness, but also improve surgical planning and promote patient-centered treatment. Such advancements have the potential to greatly improve patient outcomes, surgical planning, and diagnostic procedures.

On the other hand, the research in [16] explores the combination of machine learning and CT imaging, providing AI-driven diagnostic methodologies and individualized patient care techniques. These approaches promise to boost medical imaging studies more broadly and serve as a cornerstone for developments in disease recognition and treatment planning as well.

Table 1. Comparative analysis of the major contributions to kidney segmentation techniques in CTU imaging.

Reference	Author/s	Domain Adaptation Strategy	Methodology Highlights	Key Contributions
[8]	(Zeng et al., 2021)	Unsupervised	Utilized 2D networks for kidney segmentation from urographic images	Comparable or better performance than supervised methods on clinical urography data
[14]	(Guanyu Yang et al., 2014)	Unsupervised	Implemented automatic kidney segmentation with multi-atlas image registration	Achieved high accuracy on CT urography and CT angiographic images
[15]	(Zhao et al., 2020)	Unsupervised	Proposed MSS 3D U-Net for segmenting kidneys and kidney tumors from CT images	Enhanced performance with a connected-component-based post-processing method
[16]	(Li et al., 2022)	Deep learning	Provided open-source, unenhanced abdominal CT dataset for training deep learning networks	Accomplished highly accurate 2D and 3D segmentation of kidneys and kidney stones
[17]	(Xu & Lyu, 2016)	Unsupervised	Employed 3D-HCT and IVU for treating lower pole calyx stones	Demonstrated advantages of 3D-HCT in kidney lower pole spatial anatomy analysis
[18]	(Çiçek et al., 2016)	Unsupervised	Applied volumetric segmentation using sparsely annotated images	Achieved effective results for complex 3D structures like Xenopus kidney
[19]	(da Cruz et al., 2020)	Deep learning	Utilized image-processing techniques and deep CNNs to delimit kidneys	Aided in the early diagnosis of kidney tumors
[20]	(Cuingnet et al., 2012)	Deep learning	Combined random forests and template deformation for segmentation	Achieved rapid and accurate detection and segmentation per volume
[21]	(Thong et al., 2016)	Deep learning	Implemented patch-wise approach with a ConvNet for voxel class prediction	Enabled efficient predictions for entire CT scan volume segmentation

provides an overview of the studies that have been examined, highlighting their major approaches and contributions towards the study of the kidney segmentation in CTU imaging. Unsupervised and deep learning are the two basic categories used to categorize domain adaption techniques. The developments in these areas point to a trend toward automated and precise kidney segmentation techniques, which are useful for applications like disease diagnostics and surgical planning.

From the studies that were reviewed,

Table 2. Comparative analysis of results from various 3D kidney segmentation methodologies in CT urography imaging.

Reference	Title of the Paper	Dice Similarity Coefficient	Jaccard Index	Surface-to-Surface Distance
[14]	“Automatic kidney segmentation in CT images based on multi-atlas image registration”	95.2		0.913 mm
[15]	“MSS U-Net: 3D segmentation of kidneys and tumors from CT images with a multi-scale supervised U-Net”	MSS U-Net: 96.9 Classic 3D U-Net: 96.2	MSS U-Net: 94.1 Classic 3D U-Net: 93.0
[19]	“Kidney segmentation from computed tomography images using deep neural network”	96.33	93.02
[21]	“Convolutional networks for kidney segmentation in contrast-enhanced CT scans”	ConvNet-Coarse Left: 94.53 91.72–95.04 Right: 93.07 89.99–94.28 ConvNet-Fine: Left: 93.62 91.99–94.98 Right: 92.52 88.83–94.47
[22]	“Automatic Segmentation of Kidney and Renal Tumor in CT Images Based on 3D Fully Convolutional Neural Network with Pyramid Pooling Module”	93.1		4.21 pixels
[23]	“3D Kidney Segmentation from CT Images Using a Level Set Approach Guided by a Novel Stochastic Speed Function”	97.0

presents the results of various strategies for the challenge of kidney segmentation using CT images. Convolutional neural networks, 3D U-Nets, multi-atlas image registration, and other machine learning techniques are examples of some of these techniques. The Dice Similarity Coefficient, the Jaccard Index, and surface-to-surface distance are three measures used to assess performance. The degrees of similarity and precision within the automated segmentation outcomes generated by the algorithms and the benchmark standard are determined using these criteria.

The potential applications and subsequent clinical implications of reviewed research papers and techniques for kidney segmentation and reconstruction in CT urography imaging are given in

Table 3. Comparative analysis of methodologies for kidney segmentation and reconstruction in CT urography imaging.

Ref.	Author/s	Applications	Clinical Implications
[8]	(Zeng et al., 2021)	Kidney Disease Diagnosis, Surgical Planning, Outcome Analysis, Intuitive Visualization, Automated Workflow	Enhanced Diagnostic Accuracy, Improved Surgical Outcomes, Reduced Variability, Efficient Workflow, Facilitating Research, Patient-Centered Care
[14]	(Guanyu Yang et al., 2014)	Computer-Aided Diagnosis, Treatment Planning, Disease Monitoring, Clinical Research, Education and Training	Improved Diagnostic Accuracy, Time Efficiency, Enhanced Treatment Decisions, Minimally Invasive Interventions, Tracking Treatment Response, Clinical Workflow Integration, Patient Care Standardization
[15]	(Zhao et al., 2020)	Radiomic Analysis, Surgical Planning, Automated Segmentation	Improved Diagnostic Accuracy, Efficient Training, Enhanced Segmentation Performance, Clinical Decision Making, Standardized Segmentation, Advancements in Medical Imaging, KiTS19 Challenge
[16]	(Li et al., 2022)	Automated Kidney Segmentation, Automated Kidney Stone Detection, Radiomics and Machine Learning Analyses, Open-Source Abdominal CT Dataset	Precise and Accurate Kidney Segmentation, AI-Driven Diagnostic Strategies, Personalized Patient Care, Decision-Making Support, Advancement in Kidney Disease Management
[17]	(Xu & Lyu, 2016)	Preoperative Planning, Anatomic Assessment	Advantages of 3D-HCT, Accuracy of 3D-HCT, Improved Preoperative Evaluation, Enhanced Surgical Outcomes, Patient Safety, Individualized Treatment, Clinical Decision Making
[18]	(Çiçek et al., 2016)	Semi-Automated Segmentation, Fully Automated Segmentation	Improved Efficiency in Segmentation, Potential for Standardization, Enhanced Diagnostic Accuracy, Data Augmentation for Robustness, No Pre-trained Network Required, Potential for Other Complex Structures
[19]	(da Cruz et al., 2020)	Improved Disease Diagnosis, Enhanced Treatment Planning, Time Savings and Efficiency, Consistency in Segmentation Results	Assistance in Early Detection of Kidney Tumors, Reduction of False Positives, Facilitating Medical Research
[20]	(Cuingnet et al., 2012)	Nephrology Information Extraction, Clinical Routine Integration, Automated Diagnosis and Assessment, Treatment Planning and Follow-Up	Improved Diagnosis Accuracy, Efficient Patient Care, Enhanced Research Opportunities, Standardization of Imaging Analysis, Reduced Workload for Radiologists and Nephrologists
[21]	(Thong et al., 2016)	Guiding Patient Diagnosis, Treatment Planning, Follow-ups and Monitoring	Time-Efficient Kidney Segmentation, Improved Diagnostic Accuracy, Standardization of Kidney Segmentation, Highly Variable Dataset
[22]	(Yang et al., 2018)	Improved Surgical Planning, Automated Segmentation for Efficiency, Enhanced Diagnostic and Prognostic Tools, Standardization and Consistency, Assisting Radiomics Studies	Enhanced Patient Outcomes, Potential for Personalized Treatment, Clinical Translation and Adoption

. This encompasses a multitude of potential applications, including diagnostic tools, surgical planning, automated segmentation, disease monitoring, and more, all of which have significantly impacted various aspects of urography. The table also brings to light the profound clinical implications, including but not limited to improved diagnostic accuracy, time efficiency, enhancement of surgical outcomes, and individualized treatment options, thus reflecting the importance and extensive application of these methodologies in improving patient care and healthcare outcomes. Each entry in the table is comprehensively referenced to promote further reading and understanding of these advancements in the field of CT urography imaging.

5. Conclusions

The inference from the article review is that the domain of unsupervised adaptation for 3D segmentation in CT urography imaging showcases a dynamic and promising landscape. With continued research and innovation, as seen from the diverse studies we have reviewed, the horizon seems promising for more refined methodologies that could reshape the paradigm of medical imaging and patient care.

However, like any evolving field, there are inherent limitations. While many of these methodologies showcase impressive results, their practical implementation in diverse clinical settings remains a challenge. Factors such as variability in data acquisition, differences in CT scanner protocols, and patient demographics can influence the generalizability of some techniques. Additionally, the focus of many reviewed studies was predominantly on accuracy metrics, possibly overlooking other essential aspects like computational efficiency, ease of use, and clinical integration.

Future directions: The trajectory of unsupervised domain adaptation in CT urography imaging indicates a promising horizon. As the field matures, a shift towards amalgamating these techniques with other emergent technologies like quantum computing or augmented reality could revolutionize imaging further. Research should also prioritize the development of adaptive algorithms capable of self-optimization across diverse clinical scenarios, ensuring broader applicability. Furthermore, embracing a more interdisciplinary approach, integrating insights from radiology, data science, and even fields like neuromorphology, could pave the way for innovations that are holistic and more in tune with complex clinical demands.