TGA-GS: Thermal Geometrically Accurate Gaussian Splatting

Abstract

Novel view synthesis and 3D reconstruction have been extensively studied. Three-dimensional Gaussian Splatting (3DGS) has gained popularity due to its rapid training and real-time rendering capabilities. However, RGB imaging is highly dependent on ideal illumination conditions. In low-light situations such as at night or in the presence of occlusions, RGB images often suffer from blurred contours or even complete failure in imaging, which severely restricts the application of 3DGS in such scenarios. Thermal imaging technology, on the other hand, serves as an effective complement. Thermal images are solely influenced by heat sources and are immune to illumination conditions. This unique property enables them to clearly identify the contour information of objects in low-light environments. Nevertheless, thermal images exhibit significant limitations in presenting texture details due to their sensitivity to temperature variations rather than surface texture features. To capitalize on the strengths of both, we propose thermal geometrically accurate Gaussian Splatting (TGA-GS), a novel Gaussian Splatting model. TGA-GS is designed to leverage RGB and thermal information to generate high-quality meshes in low-light conditions. Meanwhile, given low-resolution thermal images and low-light RGB images as inputs, our method can generate high-resolution thermal and RGB images from novel viewpoints. Moreover, we also provide a real thermal imaging dataset captured with a handheld thermal infrared camera. This not only enriches the information content of the images but also provides a more reliable data basis for subsequent computer vision tasks in low-light scenarios.

1. Introduction

The field of scene reconstruction and novel view synthesis based on visible light has achieved remarkable progress, with applications spanning various domains. These applications include creating immersive experiences for users in virtual reality (VR), aiding vehicles in environmental perception and path planning for autonomous driving, assisting robots in better understanding and interacting with their environments, supporting architectural design and monitoring in the construction industry, digital twinning, and even playing crucial roles in specialized fields such as materials research and electronic component inspection.

The advent of Neural Radiance Fields (NeRFs) [1] brings new perspectives and methods to traditional 3D reconstruction and novel view synthesis. NeRFs utilize multi-layer perceptrons (MLPs) to simulate scene radiance fields through differentiable volume rendering, enabling the learning of 3D scene representations from a set of 2D images and achieving realistic view synthesis. However, NeRFs suffer from slow training and rendering speeds, which to some extent limit their applications in scenarios demanding real-time performance.

This limitation spurs the development of 3D Gaussian Splatting (3DGS) [2], which uniquely transforms the existing technical framework. By representing scenes using a multitude of 3D Gaussians, 3DGS can achieve real-time rendering while maintaining visual fidelity comparable to NeRFs, leading to a significant leap in the efficiency of 3D reconstruction and novel view synthesis. Nevertheless, most current 3DGS methods primarily process RGB image data under ideal lighting conditions. In real-world scenarios such as nighttime, low-light indoor environments, or situations with significant occlusions, RGB images face numerous challenges. Due to the photon-starved effect [3], performance significantly deteriorates under low-light conditions, leading to increased image noise, reduced contrast, blurred object contours, and sometimes the inability to acquire sufficient effective image information, greatly limiting the effectiveness and reconstruction accuracy of 3DGS technology in these special scenarios.

At the same time, thermal imaging technology, as a crucial imaging modality, offers irreplaceable advantages. Thermal images primarily reflect the temperature distribution on the surface of objects, capturing mid-wave infrared (MWIR, 3–5 $\mathsf{\mu }$m) and long-wave infrared (LWIR, 8–14 $\mathsf{\mu }$m) radiation emitted by objects based on their temperature and emissivity [4]. Unlike passive RGB sensors, thermal sensors provide illumination-invariant detection capabilities, enabling clear identification of object contours and approximate shapes even in complete darkness, smoke/fog [5], and extreme weather conditions [6], facilitating robust scene understanding. Leveraging this capability, thermal imaging can distinctly identify object contours and approximate shapes in low-light environments, providing crucial complementary information for scene understanding. With these advantages, thermal imaging technology finds wide applications in various fields. In military and defense [7,8,9,10], it aids in target detection and recognition; in post-disaster rescue scenarios [11,12,13], it helps rescue teams quickly locate survivors; in environmental protection [14,15,16,17,18], it detects the pollution status of water and space; in the construction industry [19,20,21,22,23,24,25], it detects thermal defects in building structures; in agriculture [18,26,27,28,29,30,31,32,33], it helps monitor crop growth conditions; in industry [6,34], it is used for equipment fault diagnosis; in the medical field [35,36,37], it assists in disease detection and diagnosis; and in autonomous driving [38,39], it provides additional perceptual information for vehicles under adverse lighting conditions. Additionally, thermal imaging technology also plays a significant role in computer vision tasks such as object detection and tracking [40,41,42], recognition and segmentation [43,44,45], motion estimation [46], and SLAM [47].

Despite these benefits, thermal imaging poses inherent challenges for 3D reconstruction: (1) lower spatial resolution (usually 640 × 480) compared to RGB sensors [48], (2) lack of photometric texture details due to thermal diffusion effects [49,50,51], and (3) spectral mismatch between thermal radiation peaks (8–14 $\mathsf{\mu }$m) and visible light (380–750 nm) complicating the fusion of cross-modal data [52].

In recent years, although some thermal image methods based on NeRFs have been proposed [53,54,55,56,57,58,59,60], attempting to leverage strong modeling capabilities of NeRFs to handle thermal image data, the inherent slow rendering speed and implicit scene representation of NeRFs still constrain the practical effectiveness of these methods. With the widespread application of 3DGS methods, there have also been emerging thermal image methods based on Gaussian Splatting (GS) models [61,62,63,64]. However, most of these methods mainly focus on novel view synthesis under normal lighting conditions and fail to fully exploit the potential of thermal imaging in low-light environments.

Given the current research status, our study, based on an in-depth examination and analysis of existing technologies, introduces an innovative method called thermal geometrically accurate Gaussian Splatting (TGA-GS) based on the GS model. TGA-GS can simultaneously process thermal and RGB data from thermal sensors, leveraging the complementary information from both types of data to achieve the goal of reconstructing high-quality scenes from low-light environments effectively. Notably, our method possesses unique advantages: even when provided with low-resolution thermal and RGB images under low-light conditions, it can generate high-resolution thermal and RGB images, offering a high-quality data foundation for subsequent image analysis and processing. Furthermore, to better support and validate our research, we construct an innovative dataset. This dataset includes RGB and thermal images captured in low-light environments, providing rich and valuable data resources for 3D reconstruction and novel view synthesis research.

In summary, the main contributions of our study are as follows:

• We propose the TGA-GS method, focusing on 3D thermal model reconstruction in low-light conditions. Extensive experimental results demonstrate that TGA-GS outperforms existing relevant methods in multiple evaluation metrics, showcasing its outstanding performance in 3D reconstruction under low-light environments.
• Our method exhibits robust novel view synthesis capabilities, generating high-resolution images from low-resolution inputs, including high-resolution thermal and RGB images, effectively enhancing image quality and information richness.
• We create a novel dataset that provides essential data support for 3D reconstruction and novel view synthesis research. This dataset comprises RGB and thermal images captured in low-light environments, offering a wealth of samples for studying image features and reconstruction algorithms under low lighting conditions and facilitating further advancements in related fields.

The rest of this paper is organized as follows: Section 2 reviews related work on thermal imaging and 3DGS. Section 3 provides a detailed introduction to the theoretical derivation and specific implementation of the proposed method, mainly including multimodal calibration, training phase, and loss function construction. Section 4 presents experimental results comparing TGA-GS with state-of-the-art methods on thermal images, RGB images, and 3D reconstruction. Finally, in Section 5, we summarize the work in this paper and look ahead to future work.

2. Related Work

2.1. Thermal Imaging

Thermal imaging has long been an important complement to visible light sensors, based on Planck’s blackbody radiation law [65] and the Stefan–Boltzmann equation [66], fundamentally overcoming the limitations of visible light sensors that rely on environmental lighting. Due to the technological breakthrough of uncooled microbolometer arrays [67], this technology has gradually expanded from the early military target recognition field to civilian scenarios [6]. Its unique physical perception mechanism enables the thermal imaging system to operate stably in complete darkness, through smoke/fog [5] and other obstructions and under extreme weather conditions.

This feature gives thermal imaging a unique advantage in areas such as security and surveillance, preventive maintenance, building inspections [68], monitoring rock formations [69], and more. Due to its non-contact nature, thermal imaging technology has further expanded its applications in medical diagnostics, agricultural monitoring, and archaeological research.

The hardware design of thermal imaging systems poses inherent challenges; the high cost of germanium lenses [70] and limitations in the detector unit size result in significantly lower spatial resolution of thermal sensors compared to visible light sensors. Additionally, thermal images are susceptible to the ghosting effect, where the environmental thermal radiation reflected or scattered from the object’s surface can blur the actual temperature distribution, leading to texture loss and reduced contrast [71]. To overcome these limitations, some studies propose thermal imaging super-resolution [72], while others propose multimodal fusion solutions. For instance, Maset et al. [25] and De Luis-Ruiz et al. [20] combine RGB images with thermal images using photogrammetric techniques to build high-precision 3D models, albeit relying on dense RGB-thermal image pairs for acquisition. Chen et al. [73] attempt direct 3D reconstruction using thermal images but find it only suitable for high-contrast scenes and challenging to generalize to complex environments.

In the field of sensor fusion, early methods often rely on lidar or depth cameras to provide geometric information, projecting thermal data onto 3D models, but these methods require cumbersome manual calibration and are susceptible to errors in depth measurements. In recent years, advancements in deep learning have propelled multimodal fusion. Ma et al. [74] enhance thermal image features through RGB image augmentation to improve depth perception, while Lang et al. [75] fuse inertial measurement unit (IMU) data to optimize attitude estimation. X-NeRF [53] is the first to extend Neural Radiance Fields (NeRFs) to the infrared spectrum, achieving scene modeling through joint learning of RGB–thermal image cross-spectral representations. Subsequent works expand into the thermal NeRF [54,55,56,57,58,59,60] and thermal 3DGS [61,62,63,64] domains. However, current thermal GS methods focus on normal lighting conditions and fail to utilize the advantages of thermal imaging under low-lighting conditions and cannot achieve super-resolution output of thermal images.

2.2. Three-Dimensional Reconstruction and Novel View Synthesis

Three-dimensional reconstruction technology, which retrieves the geometric information of a 3D scene from 2D images, holds significant value in the fields of visualization, measurement, and analysis. Traditional methods are mainly divided into two categories: point cloud reconstruction methods based on Structure from Motion (SfM) [76,77] and Multi-View Stereo (MVS) approaches [78] and volume modeling methods based on implicit representations such as radiance fields or signed distance fields. SfM algorithms like COLMAP recover camera poses and sparse point clouds by matching feature points across multiple views, followed by dense reconstruction using MVS. However, these methods are prone to a decrease in reconstruction quality due to feature matching failures in low-texture scenes or with low-resolution inputs [79], and they rely on rigid scene assumptions [80,81].

The advent of Neural Radiance Fields (NeRFs) [1] marks a breakthrough in implicit 3D representation, using Multi-Layer Perceptrons (MLPs) to model scene radiance fields and density fields, enabling the generation of high-quality novel views from sparse RGB images. Subsequent research focus on various directions of improvement. RawNeRF [82] and Aleth-NeRF [83] enhance reconstruction capabilities in low-light environments, W-NeRF [84] handles dynamic lighting, and NeuS [85] achieves high-fidelity surface reconstruction. Zeng et al. [86] propose a method for reconstructing power transmission lines (PTLs) based on images, and some works combine LiDAR point clouds [87] or near-infrared images [53] to enhance representation capabilities, while others use depth information to enhance effects [88].

The emergence of 3D Gaussian Splatting (3DGS) [2] revolutionizes the explicit 3D reconstruction paradigm. Due to its real-time rendering capabilities and impressive performance, 3DGS garners increasing attention. Subsequent works further explore rendering effects [89,90,91], compact representations [92], and reconstruction capabilities [93,94,95]. Additionally, Rangelov et al. [96] discuss the influence of different camera settings on the quality of 3D reconstruction, and Ye et al. [97] design a camera response module to compensate for inconsistencies in multiple views. However, the current GS reconstruction method is not ideal for 3D reconstruction of RGB or thermal images obtained in low-light environments and cannot fully utilize the complementary information of the two modalities.

3. Methods

3.1. Preliminaries

We first review 3DGS [2] and 2DGS [94]. 3DGS uses explicit 3D Gaussian points as its main rendering entity. A 3D Gaussian point, with mean vector $\mu \in {\mathbb{R}}^3$ and covariance matrix $\Sigma \in {\mathbb{R}}^{3\times 3}$, is mathematically defined as:

$$G\left(x\right)=e^{-{\displaystyle \frac{1}{2}}{(x-\mu )}^{\mathsf{T}}{\Sigma }^{-1}(x-\mu )}$$

(1)

where x is an arbitrary position within the 3D scene and the covariance matrix of the 3D Gaussian $\Sigma =RS{S}^{\mathsf{T}}{R}^{\mathsf{T}}$ is factorized into a scaling matrix S and a rotation matrix R.

2DGS adopts flat 2D Gaussians embedded in 3D space for scene representation, which is different from 3DGS. Each 2D Gaussian primitive has opacity $\alpha$ and view-dependent appearance c with spherical harmonics. For volume rendering, Gaussians sort the projected 2D Gaussians by their depth and compute the color at each pixel by front-to-back alpha blending:

$$C\left(x^{\prime }\right)=\sum _{i=1}{c}_i{\alpha }_i{\mathcal{G}}_i\left(\mathsf{u}\left(x^{\prime }\right)\right){\Pi }_{j=1}^{i-1}(1-{\alpha }_j{\mathcal{G}}_j\left(\mathsf{u}\left(x^{\prime }\right)\right)$$

(2)

where $x^{\prime }$ represents a homogeneous ray emitted from the camera and passing through uv space, while $\mathcal{G}\left(\mathsf{u}\right)=e^{-{\displaystyle \frac{u^2+v^2}{2}}}$ is 2D Gaussian value of the point $\mathsf{u}$ = $(u,v)$ in $uv$ space.

3.2. Multimodal Calibration

To address the spatial pose deviations between thermal imaging sensors and RGB sensors, as well as inherent camera distortions, we propose a multimodal joint calibration strategy based on thermal radiation modulation. Specifically, this process can be divided into three steps: calibration plate preparation, thermal gradient induction, and multimodal calibration.

Calibration Plate Preparation: We utilize a standard checkerboard calibration plate typically employed for RGB camera calibration, as shown in Figure 1. The black regions of the plate are uniformly coated with a pre-configured AgNWs/PVB ethanol solution, forming a composite coating characterized by high visible-light transmittance (~83.0%), high mid-infrared reflectivity (~69.8%), and low emissivity [98]. The thermal absorption capability of the composite structure primarily depends on the emissivity of the outermost layer [99]. Consequently, the AgNWs coating effectively suppresses thermal radiation emission in the black regions.

Thermal Gradient Induction: After allowing the prepared calibration plate to reach thermal equilibrium in a room-temperature environment (25 °C), it is rapidly transferred to a low-temperature environment of −20 °C. The uncoated white regions, due to their high emissivity, rapidly radiate heat to the environment and cool down, while the coated regions exhibit slower temperature decline owing to the low emissivity of the AgNWs/PVB composite coating, which suppresses thermal radiation, thereby creating significant temperature differences between the two areas.

Multimodal Calibration: Once substantial temperature differences are established between the black and white regions of the calibration plate, we simultaneously capture color and thermal images using thermal imaging equipment. Conventional camera calibration methods are then applied to achieve precise multimodal sensor alignment.

This methodology ensures spatiotemporal synchronization between thermal and RGB modalities while addressing intrinsic sensor distortions, providing a robust foundation for subsequent cross-modal data fusion.

3.3. TGA-GS

We designed a multimodal Gaussian training methodology, as shown in Figure 2. By deeply integrating thermal radiation physics constraints with geometric priors, our approach achieves high-precision 3D reconstruction and cross-modal super-resolution generation in low-light scenarios. Specifically, we first train a foundational Gaussian Splatting (GS) model using low-light RGB images, then fine-tune it with thermal maps. The entire training process consists of two phases: pre-training and end-to-end training.

Pre-training Phase: During this phase, we train a baseline low-light GS model using low-light RGB images from the same scenes intended for end-to-end training. Although pre-training with low-light RGB images may lead to texture detail loss and inaccurate depth estimation during training, it enables initialization based on partially observable information, thereby significantly reducing convergence time in subsequent end-to-end training.

End-to-End Training Phase: As illustrated in Figure 2, this phase fine-tunes the pre-trained low-light GS network using raw thermal maps and low-light RGB images. The outputs of the GS network are then fed into thermal SR and RGB SR modules to generate high-resolution thermal and RGB images. This stage comprises two key steps. (1) The first is the multimodal Gaussian network. As illustrated in Figure 2a, we extend the pre-trained 3DGS by jointly optimizing it with low-resolution low-light RGB and thermal images. The thermal modality provides robust geometric priors through thermal gradients (e.g., object boundaries in darkness), while the RGB modality guides texture recovery. This mutual constraint enhances the model’s ability to resolve ambiguities in low-light 3D perception. (2) The second is the thermal and RGB SR networks. As depicted in Figure 2b, the thermal maps and low-light RGB outputs from the multimodal Gaussian network are processed through dedicated thermal SR and RGB SR networks. These modules not only enhance their respective modalities but also exchange cross-modal guidance signals—thermal gradients inform texture recovery in RGB SR, while RGB edges refine thermal detail reconstruction.

3.4. Loss

For the entire training process, we designed multiple loss functions to impose constraints. Based on the characteristics of input images, we adapted the original L1 and Structural Similarity Index (SSIM) losses from the Gaussian Splatting (GS) network. When processing low-light RGB inputs, inspired by the Lit-MSE loss in Aleth-NeRF [83], we modified the standard L1 loss into an $L_{1-\mathrm{low}}$ loss to enhance adaptability to dark environments. The formulation is given by:

$$L_{1-\mathrm{low}}={∥\Phi (\widehat{C}+ϵ)-\Phi (C+ϵ)∥}_1$$

where $ϵ={10}^{-3}$ follows the configuration in Aleth-NeRF, $\Phi$ denotes the inverse tone mapping curve, C represents ground truth pixel values, and $\widehat{C}$ denotes predicted pixel values.

The SSIM loss evaluates image distortion through three components: luminance (estimated via means ${\mu }_x,{\mu }_y$), contrast (estimated via standard deviations ${\sigma }_x,{\sigma }_y$), and structure (measured via covariance ${\sigma }_{xy}$). The SSIM formula remains unchanged for thermal images, as temperature variations can still be assessed through luminance patterns at the image level.

For the final SR outputs, we designed an enhancement loss $L_{\mathrm{en}}$, combining perceptual and L1 losses:

$$L_{\mathrm{en}}={\lambda }_{\mathrm{per}}{L}_{\mathrm{per}}+{\lambda }_{\mathrm{L}1}{L}_{\mathrm{L}1}$$

where $L_{\mathrm{per}}$ is computed using a pretrained VGG19 network $\varphi$ to measure feature-level similarity between outputs O and ground truth $GT$, while ${\lambda }_{\mathrm{per}}$ and ${\lambda }_{\mathrm{L}1}$ are weighting hyperparameters.

To address geometric ambiguity in low-light 3D reconstruction, we propose a Thermal Gradient Alignment Loss. This loss leverages thermal imaging gradients to guide surface geometry optimization. Specifically, thermal edges (corresponding to temperature transitions) provide robust geometric priors under low-light conditions. The loss enforces consistency between the thermal gradient $\nabla T$ and the geometric surface normal n derived from the GS model:

$$L_{\mathrm{thermal}-\mathrm{align}}=\sum _i{∥\nabla T_i·n_i∥}_1$$

where $\nabla T_i$ is the thermal gradient at position i, and $n_i$ is the normalized surface normal. By aligning thermal and geometric boundaries, this loss mitigates depth estimation errors caused by low-contrast RGB inputs, significantly improving reconstruction fidelity for complex scenes.

4. Experiments

4.1. Datasets and Baselines

To evaluate the performance of 3D reconstruction and novel view synthesis under low-light conditions using multimodal RGB-thermal inputs, we constructed a specialized dataset captured by the commercial-grade FLIR E6 PRO handheld thermal imaging system. This advanced device simultaneously acquires aligned RGB and thermal imagery, featuring a thermal sensor with an operational range of −20 °C to 550 °C and measurement accuracy of ±2% of reading. The dataset includes five distinct indoor environments, specifically designed to simulate challenging low-light search-and-rescue scenarios. For each scene, we provide raw thermal images captured by the thermal camera, RGB images, thermal imaging data, MSX (Multi-Spectral Dynamic Imaging) images (edge-enhanced thermal–visual fusion), and camera pose data.

For comprehensive performance evaluation, we select two types of baselines for our experiments: (1) High-rendering-quality methods, used exclusively in novel view synthesis experiments, including 3DGS [2] and ThermalGaussian [62]. For ThermalGaussian, we opt for the OMMG method from the original paper due to its superior rendering quality within the ThermalGaussian framework. (2) High-reconstruction-quality methods, serving as baselines for both novel view synthesis and reconstruction experiments, including 2DGS [94] and RaDe-GS [95].

4.2. Evaluation Metrics

To evaluate the performance of our methods, we employ a set of metrics that assess quality, and memory usage.

Quality: To evaluate the quality of novel view synthesis, we compare the generated images against ground truth images captured from the corresponding target viewpoints. Specifically, the Peak Signal-to-Noise Ratio (PSNR) quantifies pixel-level differences between the synthesized novel views and the real images of those views. Higher PSNR values indicate smaller deviations from the ground truth. Additionally, the Structural Similarity Index (SSIM) [100] measures luminance, contrast, and structural consistency between the generated and real images, where higher SSIM reflects closer perceptual alignment. Finally, the Learned Perceptual Image Patch Similarity (LPIPS) [101] evaluates human-perceived visual discrepancies by leveraging deep features, emphasizing differences more relevant to human observers. Importantly, the ground truth images used for comparison are real-world captures from novel viewpoints not included in the training set, ensuring unbiased validation of our method’s ability to generalize to unseen perspectives.

Memory Usage: To assess the memory requirements of our method, we measure the memory usage in megabytes (MB). This metric provides insights into the resource demands of our approach, allowing us to optimize memory consumption and ensure efficient utilization of computational resources.

Efficiency: We measure efficiency by calculating the number of frames processed per second (FPS). This metric provides insight into the real-time performance of our approach and its ability to handle high frame rates. Higher values indicate faster processing times, which are desirable for applications requiring real-time or near-real-time performance.

4.3. Implementation Details

Our method builds upon the 3DGS framework [2], with all experimental settings strictly aligned with the reference implementation. All baseline methods and our proposed TGA-GS are trained under identical conditions on the same dataset to ensure fair comparison. Each experiment, including those for baselines, underwent 30K iterations of training on NVIDIA V100 GPU. For TGA-GS (NVIDIA, Santa Clara, CA, USA), while a pre-training phase using low-light RGB images is employed to accelerate convergence in subsequent end-to-end training, the total number of iterations (30 K) remains consistent with other methods, eliminating potential bias from extended training. The rendered RGB and thermal images share a resolution of $640\times 480$, matching the input specifications of the evaluation pipeline.

4.4. Thermal View Synthesis

To compare the quality of novel view synthesis for thermal images across various Gaussian Splatting (GS) methods, we conducted quantitative and qualitative evaluations. As shown in

Table 1. Quantitative results of thermal view synthesis (best, second best).

Metrics	Methods	Fruits1	Fruits2	Cup	Laptop	Hairdryer	Avg.
PSNR (dB)↑	3DGS [2]	37.34	29.29	29.47	35.55	30.90	32.51
	ThermalGaussian [62]	18.82	18.65	17.01	10.31	17.64	16.49
	2DGS [94]	28.52	22.79	27.77	36.12	28.48	28.74
	RaDe-GS [95]	32.56	28.46	30.24	37.25	27.75	31.25
	Ours	35.51	30.25	29.72	38.94	29.50	32.78
SSIM↑	3DGS	0.967	0.935	0.948	0.975	0.951	0.955
	ThermalGaussian	0.743	0.781	0.782	0.554	0.744	0.721
	2DGS	0.922	0.872	0.952	0.972	0.948	0.933
	RaDe-GS	0.952	0.932	0.954	0.974	0.943	0.951
	Ours	0.952	0.933	0.949	0.972	0.899	0.941
LPIPS ↓	3DGS	0.108	0.156	0.099	0.058	0.089	0.102
	ThermalGaussian	0.500	0.361	0.284	0.628	0.299	0.414
	2DGS	0.233	0.229	0.104	0.076	0.100	0.148
	RaDe-GS	0.162	0.168	0.095	0.065	0.108	0.120
	Ours	0.145	0.095	0.087	0.021	0.111	0.092
Mem. (MB) ↓	3DGS	1467	1733	1743	1783	1563	1658
	ThermalGaussian	945	1363	1179	1147	1075	1142
	2DGS	659	1069	711	711	729	776
	RaDe-GS	939	1309	1027	1859	1049	1237
	Ours	551	571	502	596	480	540
FPS ↑	3DGS	292	351	230	234	218	265
	ThermalGaussian	390	432	312	347	314	359
	2DGS	260	251	212	173	223	224
	RaDe-GS	201	298	209	140	184	206
	Ours	387	441	301	315	318	352

, our method outperforms existing approaches in visual metrics such as PSNR and LPIPS while consuming significantly less memory. Its running speed is comparable to ThermalGaussian and outperforms other methods. Specifically, our method achieves the best performance with minimal memory consumption of 540 MB, attaining a PSNR of 32.78 dB and LPIPS of 0.092. In comparison, the second-best method (3DGS [2]) exhibits similar visual metrics—with a marginal difference of 0.27 dB in PSNR and 0.01 in LPIPS—but requires nearly triple the memory footprint at 1658 MB, and more inference time.

For qualitative comparison, Figure 3 visually contrasts our method with baseline approaches across three representative scenes: Fruits2 (top row), Hairdryer (middle row), and Laptop (bottom row). The results highlight our method’s improved visual fidelity. For instance, in the Hairdryer scene, baseline methods exhibit noticeable thermal distortions such as warped nozzle geometry and blurred temperature gradients, whereas our approach preserves structural integrity and radiometric consistency. Similarly, in the Laptop scene, our method accurately reconstructs fine thermal edges including keyboard gaps that are either missing or misaligned in baseline outputs. Notably, this capability relies on the presence of temperature differences between objects. For example, the keyboard gaps in the Laptop scene are discernible only because the active electronics generate localized heat, creating measurable thermal gradients. If all objects in the scene were at equilibrium with ambient temperature (e.g., a static laptop in a thermally uniform environment), thermal imaging would fail to capture structural details such as key boundaries, as no thermal contrast would exist. These visual improvements align with the quantitative metrics, validating our method’s robustness in low-light multimodal reconstruction tasks.

4.5. RGB View Synthesis

To assess the performance of GS methods for novel RGB view synthesis under low-light conditions, we performed rigorous quantitative assessments and qualitative visual comparisons. As summarized in

Table 2. Quantitative results of RGB view synthesis (best, second best).

Metrics	Methods	Fruits1	Fruits2	Cup	Laptop	Hairdryer	Avg.
PSNR (dB)↑	3DGS [2]	37.26	36.37	38.43	41.79	36.77	38.13
	ThermalGaussian [62]	23.47	25.98	26.74	23.53	26.42	25.23
	2DGS [94]	37.55	36.21	38.11	39.85	37.18	37.78
	RaDe-GS [95]	38.58	35.75	38.37	39.70	36.49	37.78
	Ours	39.98	39.66	42.12	44.05	38.09	40.77
SSIM↑	3DGS	0.941	0.925	0.953	0.970	0.941	0.946
	ThermalGaussian	0.727	0.695	0.743	0.635	0.715	0.703
	2DGS	0.938	0.918	0.945	0.952	0.943	0.940
	RaDe-GS	0.929	0.924	0.948	0.955	0.936	0.939
	Ours	0.937	0.939	0.960	0.963	0.852	0.930
LPIPS ↓	3DGS	0.078	0.115	0.131	0.055	0.130	0.102
	ThermalGaussian	0.249	0.264	0.319	0.240	0.338	0.282
	2DGS	0.080	0.122	0.147	0.076	0.134	0.112
	RaDe-GS	0.087	0.126	0.153	0.077	0.149	0.118
	Ours	0.075	0.075	0.110	0.028	0.124	0.082
Mem. (MB) ↓	3DGS	1433	1685	1447	1511	1403	1496
	ThermalGaussian	945	1363	1179	1147	1075	1142
	2DGS	827	1089	951	1177	717	952
	RaDe-GS	865	1135	887	941	817	929
	Ours	551	571	502	596	480	540
FPS ↑	3DGS	307	428	342	333	334	349
	ThermalGaussian	442	487	358	365	380	406
	2DGS	215	239	220	152	225	210
	RaDe-GS	205	283	225	228	228	232
	Ours	439	544	505	447	445	476

, our approach surpasses existing methods in critical visual metrics, including PSNR and LPIPS, while maintaining substantially lower computational resource requirements and high computational efficiency. The quantitative analysis reveals that our method achieves state-of-the-art performance with a memory footprint of 540 MB, delivering a PSNR of 40.77 dB and LPIPS of 0.082. In contrast, the second-best performer (3DGS [2]) demonstrates comparable but inferior visual metrics (2.64 dB lower PSNR and 0.02 higher LPIPS) while consuming 1496 MB of memory (approximately three times the model’s requirements) and 349 FPS inference speed (approximately 30% slower than ours).

Figure 4 provides qualitative comparisons across three representative scenarios: Fruits2 (top row), Hairdryer (middle row), and Laptop (bottom row). These visualizations underscore our method’s enhanced fidelity in preserving fine details under illumination-starved conditions. Notably, in the Laptop scene, our approach maintains crisp structural details such as labels, whereas competing methods exhibit blurred textures and loss of high-frequency features. This performance gap aligns with the quantitative metrics, confirming our framework’s superiority in balancing reconstruction accuracy with computational efficiency.

4.6. Three-Dimensional Object Reconstruction

To evaluate the effectiveness of TGA-GS for 3D reconstruction under low-light conditions, we conducted experiments on two scenes, Fruits1 and Cup, using 2DGS [94] and RaDe-GS [95] methods to reconstruct 3D models from low-light RGB and thermal inputs. As illustrated in Figure 5 and Figure 6, the leftmost column displays the input low-light RGB and thermal images for each scene. The middle columns show reconstruction results from 2DGS and RaDe-GS, with the upper row representing RGB-only reconstructions and the lower row thermal-only reconstructions. The rightmost column presents TGA-GS’s multi-modal (RGB+thermal) reconstruction results.

The results demonstrate that TGA-GS achieves more accurate reconstructions compared to 2DGS and RaDe-GS. In the Fruits1 scene, TGA-GS successfully distinguishes individual fruits such as bananas, while other methods struggle to resolve fine-grained structures. For the Cup scene, despite using a cup filled with hot water, thermal-only reconstructions exhibit inferior performance compared to RGB-only results. This degradation may stem from the low contrast of thermal images, which introduces ambiguity in depth estimation. In contrast, TGA-GS leverages complementary information from both modalities to generate higher-quality 3D models. The fused approach effectively mitigates the limitations of individual modalities, highlighting the advantages of multimodal integration in low-light 3D reconstruction.

4.7. Ablation Study

In our ablation study, we evaluate two core components of TGA-GS: the Thermal Gradient Alignment Loss and the SR module, as demonstrated in

Table 3. Ablation study of thermal view synthesis (best, second best).

Variant	PSNR↑	SSIM↑	LPIPS↓
w/o Thermal Gradient Alignment Loss	32.22	0.937	0.144
w/o SR module	31.54	0.930	0.340
Ours	32.78	0.941	0.092

and

Table 4. Ablation study of RGB view synthesis (best, second best).

Variant	PSNR↑	SSIM↑	LPIPS↓
w/o Thermal Gradient Alignment Loss	40.71	0.924	0.100
w/o SR module	39.26	0.904	0.218
Ours	40.77	0.930	0.082

and Figure 7. Table 3 and Table 4 reveal that the SR module significantly impacts rendering quality due to its direct influence on the final output. While the Thermal Gradient Alignment Loss, as an intermediate training constraint, also affects rendering results, its performance gaps can be mitigated by the SR module during post-processing. Conversely, Figure 7 illustrates that the SR module has minimal effect on reconstruction quality, whereas the Thermal Gradient Alignment Loss plays a critical role in determining geometric fidelity. This discrepancy arises because the SR module operates as a post-processing component, leaving the core reconstruction process of TGA-GS largely unaffected. In contrast, the Thermal Gradient Alignment Loss imposes structural constraints during training, directly shaping the 3D geometry optimization.

4.8. Practical Implications

Search and Rescue Teams: In post-disaster search and rescue operations (such as locating survivors in smoke-filled environments), when the visibility of the environment is low, TGA-GS can use RGB and thermal images collected from limited perspectives to construct a search and rescue environment. The thermal gradient can reliably outline the danger of people and structures, while the RGB image and SR module can further recover key details.

Industrial Practitioners: The perception ability of autonomous vehicles, unmanned aerial vehicles, etc., in low-light or severe conditions is enhanced. The use of TGA-GS cross-mode alignment can ensure the synchronous perception of thermal barriers (such as pedestrians) and RGB traffic signs. Meanwhile, super-resolution of low-resolution thermal input can detect distant obstacles without affecting latency.

Industrial and Building Inspectors: The multimodal calibration method proposed in this paper can better align the relevant information between thermal and RGB images and associate thermal anomaly areas (such as thermal leaks) with RGB texture details (such as rust or cracks). Meanwhile, by combining the guidance super-resolution module of TGA-GS, the quality of detection can be further improved.

5. Conclusions

In this paper, we present TGA-GS, a method for novel view synthesis and 3D reconstruction utilizing thermal and low-light RGB images. Our approach not only produces high-quality renderings from low-resolution RGB and thermal inputs but also enhances reconstruction accuracy in low-light environments while significantly reducing memory footprint during runtime. However, it is important to acknowledge certain limitations of our method. Although TGA-GS generates improved RGB outputs and 3D reconstructions, the rendered images remain low light in appearance, and reconstruction quality degrades under low-thermal-contrast conditions. Future work will focus on developing an extended framework capable of generating high-quality natural-light RGB images and further improved reconstruction results from thermal and low-light RGB inputs.