Dimma: Semi-Supervised Low-Light Image Enhancement with Adaptive Dimming

Abstract

Enhancing low-light images with natural colors poses a challenge due to camera processing variations and limited access to ground-truth lighting conditions. To address this, we propose Dimma, a semi-supervised approach that aligns with any camera using a small set of image pairs captured under extreme lighting conditions. Our method employs a convolutional mixture density network to replicate camera-specific noise present in dark images. We enhance results further by introducing a conditional UNet architecture based on user-provided lightness values. Trained on just a few real image pairs, Dimma achieves competitive results compared to fully supervised state-of-the-art methods trained on large datasets.

1. Introduction

The low-light image enhancement task aims to replicate the appearance of a photo taken with a longer exposure by reducing noise, increasing brightness, and preserving natural colors. Many research papers [1,2,3,4,5] have reported promising results on paired datasets containing well-lit and dark images. However, the effectiveness of models trained on one dataset may not generalize well to images from different cameras. Variations in how individual cameras process images pose a challenge to achieving consistent and reliable enhancement across diverse camera models. Creating a comprehensive set of paired images for numerous devices is not only challenging but also expensive, as it involves taking two images of the same scene with different exposure times, requiring static scenes and precision in capturing the photos in order to align the images. This slow process results in datasets with limited and very similar pairs, often taken by the same camera in the same location, such as the LOL dataset [1] with only 480 training pairs.

On the other hand, unsupervised methods [6,7] do not require a paired dataset in the training process; therefore, they are not biased towards limited training data and can generalize better. However, lacking defined ground-truth in the training process often leads to lower-quality results obtained by unsupervised methods.

To fill this gap, we propose a semi-supervised approach (Figure 1) that can leverage a small set of real paired images to construct a dimming module capable of replicating the camera-specific dimming process. In this way, we reduce the time needed to collect comprehensive datasets for different cameras. By utilizing a mixture density network and illumination statistics extracted from the real image pairs, the dimming module can effectively darken an image and mimic the color distortion associated with very dark images from a selected source.

Next, we create dimmed versions of images from various computer vision datasets using the constructed dimming module and train a UNet [8] model to restore their original light conditions. Our UNet architecture incorporates a brightness level conditioning to represent light differences and predict the contrast between dark and light images, rather than directly predicting the light values themselves. This approach allows the model to learn conditioned image light enhancement, providing flexibility and adaptability to various lighting conditions.

Our proposed approach offers a practical and cost-effective solution for training image enhancement models using only few paired examples. The utilization of a dimming module based on a few real paired images enables the model to mimic the context-specific dimming process accurately. Furthermore, the ability to adjust the brightening factor provides additional control over the image enhancement process, which is not feasible when relying solely on paired datasets.

Through extensive experimentation and evaluation, we demonstrate the effectiveness of our approach in enhancing low-light images while preserving natural colors. By addressing the challenges posed by camera variations, the limited availability of paired datasets, and the control over the brightening factor, our research contributes to advancing the field of low-light image enhancement and provides a practical and adaptable solution for real-world applications.

In summary, our contributions can be outlined as follows:

• We introduce a dimming module based on a mixture density network [9] and illumination statistics calculated from a small set of ground-truth image pairs. This approach replicates the color distortion and darkening process of dark images and enables semi-supervised learning of the brightening module using as few as three image pairs.
• We propose to use UNet [8] conditioned on light differences to predict the residuals between dark and light images rather than the light values themselves.
• We introduce two datasets: the FewShow-Dark (FS-Dark) dataset containing a few real image pairs captured by mobile phone cameras and the MixHQ dataset, which comprises high-quality images selected from various datasets, including COCO [10], ImageNet [11], Clic [12], Inter4K [13], and LOL [1]. Combined with our dimming module, MixHQ allows us to create artificial training pairs for semi-supervised low-light image enhancement.

2. Related Work

Low-light image enhancement is a widely recognized task in the field of computer vision, attracting significant attention from researchers. Over the years, various classic approaches have emerged, including techniques such as histogram equalization, gamma correction, and more complex algorithms such as LIME (Low-light Image Enhancement) [14]. These classic methods are known for their remarkable speed and generalizability. However, they often produce images of relatively lower quality compared to more recent deep learning techniques.

Deep learning-based approaches for low-light image enhancement have gained popularity due to their ability to generate visually appealing images. Several methods [1,2,3,4] have achieved notable results in this domain. In RetinexNet [1], the authors proposed to use a neural network for retinex decomposition. The model is trained to reconstruct both reflectance and illumination to match the well-lit images’ maps. KinD and KinD++ [2,3] share a similar idea but use separate modules for each of the maps.

One of the recent approaches, LLFlow [4], utilizes a normalizing flow that learns a distribution of images under normal light conditions given a low-light one. In addition, an encoder is used to extract an illumination-invariant color map that is then injected into the normalizing flow. Authors in [15] propose to adjust the diffusion model for low-light image enhancement. The model utilizes sampling in a pyramid resolution style by progressively increasing resolution in one reverse process. Moreover, this approach uses a global corrector to alleviate the global degradation that may occur in the reverse process. The diffusion-based approach proposed in [16] boosts the performance of low-light enhancement by regularizing the ODE trajectory. Authors in [17] postulate using a wavelet-based conditional diffusion model (WCDM) that leverages the generative power of diffusion models to produce results with satisfactory perceptual fidelity.

In most of the approaches [1,2,3], authors use a UNet [8] convolutional architecture in which the skip connections allow for modeling features of lower frequencies, producing results of higher visual quality. In [5,18], researchers adopt transformer architectures rather than relying solely on convolutional networks. LLNeRF [19] employs the Neural Radiance Fields architecture to improve the visibility of dark scenes by using multiple views of the object.

Most of the previously mentioned approaches rely on datasets consisting of paired images in which one image is captured under low-light conditions, and the other is a corresponding well-lit reference. By learning the mapping from dark to well-lit images, these models can enhance low-light images effectively. However, it is important to note that these paired datasets represent the distribution specific to the cameras used, limiting the generalization of these approaches to other camera models.

Despite the challenges associated with low-light image enhancement, there have been attempts to develop unsupervised approaches using deep learning techniques. For example, EnlightenGAN [6] proposes to leverage a large dataset of unpaired light and dark images and employs adversarial training to enhance low-light images. Another unsupervised method [7] utilizes the observation that the feature maps of trained neural networks are very similar for natural images and for the histogram equalization of the corresponding dark images. Zero-DCE [20] is a zero-shot approach, which formulates low-light image enhancement through monotonic curve estimation with many carefully designed constraints that do not rely on paired or unpaired datasets.

While unsupervised models offer flexibility and the potential to generalize to different cameras, due to the inherent nature of adversarial training, they often fall short in terms of producing high-quality images compared to approaches that employ ground-truth-based loss functions. The lack of explicit supervision in unsupervised methods makes achieving the desired level of image quality and natural color preservation challenging.

LED [21] is a few-shot method for low-light enhancement, which utilizes artificial dark images with synthetic noise to transfer the knowledge of enhancing real dark images. However, this method works only with raw images, whose noise characteristics are specific to sensor errors, while RGB images are post-processed with different pipelines for which we cannot assume any prior knowledge. DRBN [22] aims to make use of both paired and unpaired data by introducing a semi-supervised approach. Initially, it learns to predict a trajectory of corrections through a recursive architecture. Subsequently, a model that combines all corrections into the final prediction is trained in an adversarial manner, eliminating the need for paired images in the second phase. However, the training of the first part of the model still necessitates an extensive paired dataset, given the challenging nature of the task and the complexity of the architecture.

To fill the gap between approaches requiring large paired datasets and fully unsupervised methods, we propose a framework that requires only a few examples with well-fit references to achieve competitive results for the models that require a complete set of pairs for training.

3. Our Method

This section provides a detailed description of Dimma, a novel model for enhancing low-light images. Our approach comprises two key components: dimming and brightening modules (see Figure 1). The first module mimics color distortion made by specific cameras to create darkened images. These are further used to train the brightening module represented by the UNet network responsible for restoring the original image.

3.1. Dimming Module

Retinex decomposition. Previous work [1,2,3] has shown that the retinex decomposition of an image $\mathbf{I}$ to illumination $\mathbf{L}$ and reflectance $\mathbf{R}$ helps models achieve a better quality of restored images. However, most of the works attempt to do this using neural networks, making it slow and introducing reconstruction loss. On top of that, to train such a network, an extensive paired dataset is required. Our approach is based on channel-wise normalization introduced in [4] in which the i-th channel of color map $\mathbf{R}$ can be calculated as:

$${\mathbf{R}}_i={\mathbf{I}}_i\oslash \left({\displaystyle \frac{1}{3}}\sum _{i=1}^3{\mathbf{I}}_i\right),$$

(1)

where ⊘ represents Hadamard (element-wise) division, ${\mathbf{I}}_i$ and ${\mathbf{R}}_i$ represent the i-th channel of image $\mathbf{I}$ and color map $\mathbf{R}$, respectively. Assuming that the color map is a light-invariant reflectance known from retinex theory and taking into account that the image is a Hadamard product of reflectance and illumination, we obtain the illumination equal $\mathbf{L}=\mathbf{I}\oslash \mathbf{R}$.

Dimming process. In Figure 2, we present the dimming procedure that creates the corresponding dark images essential for training the brightening module in the second stage. The initial photo is split into illumination $\mathbf{L}$ and color map $\mathbf{R}$ components using retinex decomposition described in the previous paragraph. Next, illumination dimming is applied on $\mathbf{L}$ to create the darker equivalent ${\mathbf{L}}_D$. This process involves mean and deviations of mapping between light and dark illumination values calculated from a few training pairs. This process is described in the Appendix A. To align the dimming process to specific camera conditions, we utilize a Mixture Density Network (MDN), which is responsible for adjusting color map $\mathbf{R}$. This module is trained in supervised mode using a few labeled examples, for which the corresponding ground-truth color map is known. The output dark image ${\mathbf{I}}_D$ is created by the application of the Hadamard product between dark illumination ${\mathbf{L}}_D$ and color map ${\mathbf{R}}_D$ sampled from the MDN component ${\mathbf{I}}_D={\mathbf{R}}_D\odot {\mathbf{L}}_D$.

Mixture Density Network (MDN). The architecture of the MDN network is provided in Figure 3. This component is responsible for calibrating the color map crucial for creating a darker image. We suggest using a probabilistic model for the following reasons. First, we aim to mimic the noise produced by the camera on the same scene in low-light conditions. The MDN network is capable of modeling complex camera-specific noise, as shown in the Appendix D in Figure A1. Second, we train this component using only a few examples, so the precise color map prediction may be difficult to obtain using limited data. Third, the randomness injected into the process of generating dark images serves as a regularization technique for the brightening module. The ablation study discussed in details in Section 4 shows the rationality of our approach.

The MDN module takes color map $\mathbf{R}$ from the light image $\mathbf{I}$ and illuminations $\mathbf{L}$ and ${\mathbf{L}}_D$ from both light and dark images. This enables the network to learn color noise, considering the local brightness of the source and target image. This approach is motivated by the observation that dark regions in photos tend to be more noisy. The color map $\mathbf{R}$ and illuminations $\mathbf{L}$ and ${\mathbf{L}}_D$ are concatenated to create 5-channel input. To make the model as general as possible and to avoid overfitting while training on only a few images, we use only 1 × 1 convolution filters in the MDN architecture. Practically, it means that we process 5-dimensional representations for each pixel independently, with the MLP module sharing the weights across the pixels. Let ${\mathbf{x}}_{i,j}=[{\mathbf{r}}_{i,j},l_{i,j},l_{D,i,j}]$ denote the 5-dimensional representation that concatenates the pixel values at $(i,j)$ location from input color map $\mathbf{R}$ and illuminations $\mathbf{L}$ and ${\mathbf{L}}_D$, respectively. The distribution over $r_{D,i,j,k}$ representing the scalar element of ${\mathbf{R}}_D$ on the $(i,j)$-th position in channel k can be described as:

$$\begin{array}{c}\hfill p\left(r_{D,i,j,k}\right|{\mathbf{x}}_{i,j};\mathit{\theta })=\sum _{m=1}^M{\pi }_{k,m}({\mathbf{x}}_{i,j};\mathit{\theta }){\varphi }_{D,i,j,k,m},\end{array}$$

(2)

where M is the number of components in a mixture of Gaussians, $\mathit{\theta }$ represents the parameters of the MDN network, and the density of a single Gaussian ${\varphi }_{D,i,j,k,m}$ is defined as:

$$\begin{array}{c}\hfill \mathcal{N}(r_{i,j,k}+{\mu }_{k,m}({\mathbf{x}}_{i,j};\mathit{\theta }),{\sigma }_{k,m}({\mathbf{x}}_{i,j};\mathit{\theta })),\end{array}$$

(3)

where ${\pi }_{k,m}({\mathbf{x}}_{i,j};\mathit{\theta })$, ${\mu }_{k,m}({\mathbf{x}}_{i,j};\mathit{\theta })$, and ${\sigma }_{k,m}({\mathbf{x}}_{i,j};\mathit{\theta })$ are the parameters of Gaussian mixture predicted by the MDN. Each color channel has its own head, producing means ${\mu }_{k,m}({\mathbf{x}}_{i,j};\mathit{\theta })$, standard deviations ${\sigma }_{k,m}({\mathbf{x}}_{i,j};\mathit{\theta })$, and mixing coefficients ${\pi }_{k,m}({\mathbf{x}}_{i,j};\mathit{\theta })$. The means produced by the model are added to the original color map to make the network predict color differences instead of the total color map. In this way, we make the task easier for the MDN because it was trained on the limited data, which do not contain most of the hues. The training dataset size was also the reason we trained the model to predict color maps instead of entire dark images, which makes it more invariant to illumination conditions in the training data. The joined distribution over ${\mathbf{R}}_D$ can be defined as:

$$p\left({\mathbf{R}}_D\right|\mathbf{R},\mathbf{L},{\mathbf{L}}_D;\mathit{\theta })=\prod _{i,j,k}p\left(r_{D,i,j,k}\right|{\mathbf{x}}_{i,j};\mathit{\theta }).$$

(4)

Assuming access to few (K) labeled training examples ${\mathcal{D}}_K={\left\{({\mathbf{I}}^{\left(n\right)},{\mathbf{I}}_D^{\left(n\right)})\right\}}_{n=1}^K={\left\{({\mathbf{R}}^{\left(n\right)},{\mathbf{L}}^{\left(n\right)},{\mathbf{R}}_D^{\left(n\right)},{\mathbf{L}}_D^{\left(n\right)})\right\}}_{n=1}^K$ composed of light ${\mathbf{I}}^{\left(n\right)}$ and corresponding dark images ${\mathbf{I}}_D^{\left(n\right)}$, we optimize the $\mathit{\theta }$ parameters of the MDN by minimizing the conditional negative log-likelihood:

$${\mathit{\theta }}^{\mathbf{*}}=arg\underset{\mathit{\theta }}{min}\sum _n\sum _{i,j,k}-logp\left(r_{D,i,j,k}^{\left(n\right)}\right|{\mathbf{x}}_{i,j}^{\left(n\right)};\mathit{\theta }),$$

(5)

where $p\left(r_{D,i,j,k}^{\left(n\right)}\right|{\mathbf{x}}_{i,j}^{\left(n\right)};\mathit{\theta })$ is given by Equation (2). The vector ${\mathbf{x}}_{i,j}^{\left(n\right)}=[{\mathbf{r}}_{i,j}^{\left(n\right)},l_{i,j}^{\left(n\right)},l_{D,i,j}^{\left(n\right)}]$ concatenates the pixel values at $(i,j)$ location from the n-th example color map ${\mathbf{R}}^{\left(n\right)}$ and from illuminations ${\mathbf{L}}^{\left(n\right)}$ and ${\mathbf{L}}_D^{\left(n\right)}$.

3.2. Brightening Module

Once the dimming module has been trained, it can be utilized for unsupervised training of a brightening model, as depicted in Figure 1. We chose to use UNet [8], as it proved to be a very good architecture for the low-light image enhancement task [2,3,23]. We decided to further improve it by using conditioning known from diffusion models [24,25,26] to give the model information about the desired target lightness. By injecting lightness into feature maps, the model acquires a lot of information regarding the image’s final appearance. Consequently, it can generate more visually realistic images across various brightness levels. Previous works attempted to achieve this by scaling the image illumination by the desired lightness [2,4,5,15,16,23], which is a more straightforward approach but could not lead to visually appealing images with varying brightness levels that differ from those present in the training set. The detailed architecture of our brightening module is shown in Figure 4.

Let $U_{\mathit{\beta }}$ be a brightening UNet network with parameters $\mathit{\beta }$. Inspired by [4], our model takes the concatenation of the original dark image ${\mathbf{I}}_D$, its histogram equalization $H\left({\mathbf{I}}_D\right)$, color map ${\mathbf{R}}_D$, and illumination as input ${\mathbf{L}}_D$. Compared to reference approaches, our model also takes a lightness degree $\Delta m$ that controls the light level of the output image. This information is injected into UNet in a similar way as time embedding is used in diffusion models. Thanks to this approach, we can control the level of lightness while enhancing the low-light image. The model returns the residual map $\mathbf{U}$ that is further added to the dark image ${\mathbf{I}}_D$ to obtain the output image $\mathbf{I}={\mathbf{I}}_D+\mathbf{U}$. We use sigmoid activation functions on the output of $U_{\mathit{\beta }}$, as this aligns with the intuition that the model should only increase the image’s brightness.

The model $U_{\mathit{\beta }}$ is trained using the set of light images and the set of the few real image pairs ${\mathcal{D}}_N\cup {\mathcal{D}}_K$, where ${\mathcal{D}}_N={\left\{{\mathbf{I}}^{\left(n\right)}\right\}}_{n=1}^N={\left\{({\mathbf{R}}^{\left(n\right)},{\mathbf{L}}^{\left(n\right)})\right\}}_{n=1}^N$, and where ${\mathcal{D}}_K$ contains the few image pairs used to train the dimming module. We choose a pair from ${\mathcal{D}}_K$ with the probability p; otherwise, we select the light image from ${\mathcal{D}}_N$. In this case, we do not have access to illumination ${\mathbf{L}}_D$. We solve this issue by calculating ${\mathbf{L}}_D=\mathbf{\Phi }\odot \mathbf{L}$, where components of dimming matrix $\mathbf{\Phi }$ are sampled from the normal distribution $\mathcal{N}(\gamma ·{\mu }_{l_{i,j}},\alpha ·{\sigma }_{l_{i,j}}^2)$, where $l_{i,j}$ represent the element in light illumination $\mathbf{L}$. The values of ${\mu }_k$ and ${\sigma }_k$ are means and deviations for each light value calculated from the few training pairs ${\mathcal{D}}_K$, where $k\in \{0,\dots ,255\}$. The detailed estimation procedure is provided in Appendix A. In addition, we propose to scale the Gaussian means (Equation (2)) by parameter $\gamma$, where the values are sampled from the uniform distribution $\gamma \sim \mathcal{U}(a_0,a_1)$. Thanks to this, we achieve diverse levels of darkness for output images. Second, we utilize the temperature trick usually applied in generative models [27] by scaling the variances from Equation (2) by $\alpha <1$. The light image ${\mathbf{I}}^{\left(n\right)}$ together with generated dark image ${\tilde{\mathbf{I}}}_D^{\left(n\right)}$ sampled from the MDN representing the dimming module is further used as one of the training pairs to estimate the parameters $\mathit{\beta }$ of the brightening component. The model $U_{\mathit{\beta }}$ requires generated dark image ${\tilde{\mathbf{I}}}_D^{\left(n\right)}$, the corresponding histogram equalization $H\left({\tilde{\mathbf{I}}}_D^{\left(n\right)}\right)$, color map ${\tilde{\mathbf{R}}}_D^{\left(n\right)}$, and illumination as input ${\tilde{\mathbf{L}}}_D^{\left(n\right)}$. In addition, the parameter $\Delta m^{\left(n\right)}$ that controls the light level should be provided to the model. During training, the $\Delta m^{\left(n\right)}$ is calculated as the difference between the mean lightness of the original light image ${\mathbf{I}}^{\left(n\right)}$ and generated dark image ${\tilde{\mathbf{I}}}_D^{\left(n\right)}$. In the inference mode, the user provides a value to achieve different levels of light. The total loss used to train the model $U_{\mathit{\beta }}$ is represented by the sum of the mean squared error ${\mathcal{L}}_{MSE}$ and the perceptual loss ${\mathcal{L}}_P$:

$$\begin{array}{cc}\hfill {\mathcal{L}}_{MSE}+{\mathcal{L}}_P=& \hfill \sum _{n=1}^N\left|\right|{\mathbf{I}}^{\left(n\right)}-({\tilde{\mathbf{I}}}_D^{\left(n\right)}+{\mathbf{U}}^{\left(n\right)}){\left|\right|}_2^2+\hfill \\ \hfill & \hfill \sum _{n=1}^N\lambda \left|\right|f\left({\mathbf{I}}^{\left(n\right)}\right)-f({\tilde{\mathbf{I}}}_D^{\left(n\right)}+{\mathbf{U}}^{\left(n\right)}){\left|\right|}_2^2,\hfill \end{array}$$

(6)

where ${\mathbf{U}}^{\left(n\right)}=U_{\mathit{\beta }}({\tilde{\mathbf{I}}}_D^{\left(n\right)},H\left({\tilde{\mathbf{I}}}_D^{\left(n\right)}\right),{\tilde{\mathbf{R}}}_D^{\left(n\right)},{\tilde{\mathbf{L}}}_D^{\left(n\right)},\Delta m^{\left(n\right)})$ is a residual map returned by UNet for a given dark image ${\tilde{\mathbf{I}}}_D^{\left(n\right)}$, $\lambda$ is hyperparameter scaling perceptual loss, and $f(·)$ is the non-trainable feature extractor for perceptual loss. Note that ${\tilde{\mathbf{I}}}_D^{\left(n\right)}$ represents a different sample from the dimming module for each of the epochs.

Inference. After the training process, the dimming module with the MDN is discarded, and only the brightening module is used. Different light settings are controlled using a light condition value from $[0,1]$. In the validation procedure, we follow previous works [2,3,4,5,15,16,17] and condition the model using the mean pixel value derived from the ground-truth image.

4. Experiments

We conducted a series of experiments aiming to demonstrate the superiority of Dimma over previous methods: standard quantitative and qualitative comparisons, a generalizability test, training with limited data, and light-level conditioning. We adopt the experimental setup choices outlined in LLFlow [4]. We compared our model with RetinexNet [1], EnlightenGAN [6], DRBN [22], KinD++ [3], Zero-DCE [20], HEP [7], SNR-Net [18], LLFlow [4], Retinexformer [5], PyDiff [15], and GSAD [16]. For the evaluation of results, we employed PSNR, SSIM [28] on both grayscale and RGB, LPIPS [29,30], and DeltaE and NIQE [31] metrics. We also performed extensive significance tests on the obtained results. Our methodology is described in Appendix C, and the results are presented in

Table A2. Statistical significance of the differences between Dimma trained on 3, 5, and 8 pairs and other low-light image enhancement methods. Test were performed on the LOL and VE-LOL datasets, and the results are separated by slashes. The p-values were obtained using the Wilcoxon signed-rank test. The p-values are colored in green if Dimma is significantly better, red if Dimma is significantly worse, and gray if there is no significant difference. We chose the significance level of 0.01.

Dimma 3 Pairs
Methods	PSNR	SSIM	RGB-SSIM	LPIPS	DeltaE	NIQE
Zero-DCE	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
EnlightenGAN	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
HEP	<0.01/<0.01	0.126/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	0.823/<0.01
Dimma 5 pairs	0.017/0.066	0.763/0.259	<0.01/<0.01	<0.01/0.069	0.150/<0.01	0.225/0.062
Dimma 8 pairs	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/0.280	0.072/0.396
RetinexNet	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
KinD++	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	0.225/<0.01	<0.01/<0.01
SNR-Net	0.553/0.508	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	0.014/<0.01	<0.01/<0.01
LLFlow	0.028/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
Retinexformer	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/0.746
PyDiff	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
GSAD	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
Dimma 5 Pairs
Methods	PSNR	SSIM	RGB-SSIM	LPIPS	DeltaE	NIQE
Zero-DCE	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
EnlightenGAN	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
HEP	<0.01/<0.01	0.134/<0.01	0.016/<0.01	<0.01/<0.01	<0.01/<0.01	0.671/<0.01
Dimma 3 pairs	0.017/0.066	0.763/0.259	<0.01/<0.01	<0.01/0.069	0.150/<0.01	0.225/0.062
Dimma 8 pairs	0.225/<0.01	<0.01/0.011	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	0.072/<0.01
RetinexNet	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
KinD++	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	0.107/<0.01	<0.01/<0.01
SNR-Net	0.929/0.258	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	0.061/<0.01	<0.01/<0.01
LLFlow	0.538/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
Retinexformer	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/0.123
PyDiff	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
GSAD	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
Dimma 8 Pairs
Methods	PSNR	SSIM	RGB-SSIM	LPIPS	DeltaE	NIQE
Zero-DCE	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
EnlightenGAN	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
HEP	<0.01/<0.01	0.346/<0.01	0.414/<0.01	<0.01/<0.01	<0.01/<0.01	0.401/<0.01
Dimma 3 pairs	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/0.280	0.072/0.396
Dimma 5 pairs	0.225/<0.01	<0.01/0.011	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	0.072/<0.01
RetinexNet	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
KinD++	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
SNR-Net	0.712/0.090	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	0.160/<0.01	<0.01/<0.01
LLFlow	0.884/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
Retinexformer	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/0.302
PyDiff	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01
GSAD	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01	<0.01/<0.01

. Finally, we empirically justified our design choices regarding the dimming module and final fine-tuning of Dimma.

Quantitative. The first experiment involved a quantitative comparison on the LOL dataset [1]. In the first phase of semi-supervised training, we used 3, 5, or 8 real image pairs from the LOL dataset to train the dimming module. For the second phase (see Figure 1), we introduced the MixHQ dataset, which contained more than 15,000 natural images from COCO2017 [10], ImageNet [11], Clic [12], Inter4K [13], and light images from the train split of the LOL dataset. We selected good-quality images only, using the rules described in Appendix B. Those were used as unlabeled samples for the training of the brightening module. For validation, we used 5 pairs from LOL, and we tested all models on 15 pairs from the LOL test split.

As depicted in Table 1, our semi-supervised models trained on a few real image pairs surpass the unsupervised methods by a large margin and closely approach the fully supervised methods that were trained on many image pairs. This introduces a new trade-off between the number of pairs and the quality of generated images. For visual comparison, see Figure 5, Figure A3, Figure A4, Figure A5, Figure A6 and Figure A7 in Appendix E showing the results obtained for various images, including extremely dark or high-contrast ones. To measure the significance of the results, we performed statistical tests using the PSNR metric between HEP [7] and Dimma trained on 3, 5, and 8 pairs, and on LOL [1] and VE-LOL [22]. Our method demonstrates superior performance with a highly significant p-value of less than $1\times {10}^{-7}$, indicating an extremely low probability of error.

Additionally, we measured the latency of our approach together with a selected group of reference methods. The test was performed on the Kaggle platform with P100 GPU, and we tested all methods compatible with the environment. The results depicted in Table 2 demonstrate the superiority of simple unsupervised methods in latency, with Dimma being a faster alternative compared to more complex supervised approaches.

Generalizability. We proceeded with a cross-dataset comparison on the VE-LOL dataset [32], focusing exclusively on models trained on the LOL dataset and unsupervised methods. This particular experiment aimed to evaluate the ability of our method to generalize to unseen image sources. We used 100 real and 100 synthetic pairs from VE-LOL as a test set and evaluated the same methods mentioned in the previous paragraph.

Table 1. Quantitative comparisons of the LOL and VE-LOL datasets separated by slashes. All of the models were trained on LOL. The best results are in bold, and the arrows indicate whether the metric should be minimized or maximized. In order to obtain all metrics, we validate each method using code provided on GitHub repositories, and we follow the LLFlow [4] setup. The table is divided into three sections: zero-shot and unsupervised, semi-supervised, and fully-supervised. *—DRBN results were taken from [33] due to problems with running the code provided by the authors.

Method	PSNR ↑	SSIM ↑	RGB-SSIM ↑	LPIPS ↓	DeltaE ↓	NIQE ↓	Train Pairs ↓
Zero-DCE	14.86/16.07	0.65/0.58	0.56/0.47	0.34/0.48	18.82/18.51	8.22/8.92	0
EnlightenGAN	17.48/16.90	0.70/0.62	0.65/0.56	0.32/0.43	14.48/16.45	4.89/6.88	0
HEP	20.23/16.55	0.84/0.69	0.79/0.67	0.17/0.36	11.20/15.77	3.30/5.00	0
DRBN *	15.12/-	0.472/-	-/-	0.316/-	-/-	-/-	689
Dimma 3 pairs	24.44/23.20	0.84/0.85	0.78/0.81	0.23/0.22	7.91/9.07	3.29/3.66	3
Dimma 5 pairs	24.93/23.25	0.85/0.85	0.78/0.81	0.22/0.22	7.64/9.30	3.25/3.70	5
Dimma 8 pairs	25.21/23.41	0.86/0.85	0.80/0.82	0.20/0.22	7.23/8.99	3.21/3.68	8
RetinexNet	16.77/14.68	0.52/0.46	0.42/0.36	0.47/0.65	15.89/20.98	9.73/10.36	485
KinD++	21.80/20.41	0.88/0.79	0.83/0.76	0.16/0.28	8.50/11.37	4.00/4.47	460
SNR-Net	24.61/22.93	0.90/0.76	0.84/0.70	0.15/0.32	6.85/11.32	4.02/4.14	485
LLFlow	25.19/22.38	0.93/0.73	0.86/0.69	0.11/0.32	6.40/11.13	4.08/5.83	485
Retinexformer	27.18/24.85	0.90/0.87	0.85/0.84	0.13/0.18	5.52/7.74	2.95/3.62	485
PyDiff	27.17/27.07	0.93/0.91	0.88/0.88	0.10/0.15	5.33/6.98	4.01/4.46	485
GSAD	27.69/24.85	0.92/0.87	0.88/0.85	0.09/0.15	5.36/7.78	4.14/4.92	485

Table 1. Quantitative comparisons of the LOL and VE-LOL datasets separated by slashes. All of the models were trained on LOL. The best results are in bold, and the arrows indicate whether the metric should be minimized or maximized. In order to obtain all metrics, we validate each method using code provided on GitHub repositories, and we follow the LLFlow [4] setup. The table is divided into three sections: zero-shot and unsupervised, semi-supervised, and fully-supervised. *—DRBN results were taken from [33] due to problems with running the code provided by the authors.

Method	PSNR ↑	SSIM ↑	RGB-SSIM ↑	LPIPS ↓	DeltaE ↓	NIQE ↓	Train Pairs ↓
Zero-DCE	14.86/16.07	0.65/0.58	0.56/0.47	0.34/0.48	18.82/18.51	8.22/8.92	0
EnlightenGAN	17.48/16.90	0.70/0.62	0.65/0.56	0.32/0.43	14.48/16.45	4.89/6.88	0
HEP	20.23/16.55	0.84/0.69	0.79/0.67	0.17/0.36	11.20/15.77	3.30/5.00	0
DRBN *	15.12/-	0.472/-	-/-	0.316/-	-/-	-/-	689
Dimma 3 pairs	24.44/23.20	0.84/0.85	0.78/0.81	0.23/0.22	7.91/9.07	3.29/3.66	3
Dimma 5 pairs	24.93/23.25	0.85/0.85	0.78/0.81	0.22/0.22	7.64/9.30	3.25/3.70	5
Dimma 8 pairs	25.21/23.41	0.86/0.85	0.80/0.82	0.20/0.22	7.23/8.99	3.21/3.68	8
RetinexNet	16.77/14.68	0.52/0.46	0.42/0.36	0.47/0.65	15.89/20.98	9.73/10.36	485
KinD++	21.80/20.41	0.88/0.79	0.83/0.76	0.16/0.28	8.50/11.37	4.00/4.47	460
SNR-Net	24.61/22.93	0.90/0.76	0.84/0.70	0.15/0.32	6.85/11.32	4.02/4.14	485
LLFlow	25.19/22.38	0.93/0.73	0.86/0.69	0.11/0.32	6.40/11.13	4.08/5.83	485
Retinexformer	27.18/24.85	0.90/0.87	0.85/0.84	0.13/0.18	5.52/7.74	2.95/3.62	485
PyDiff	27.17/27.07	0.93/0.91	0.88/0.88	0.10/0.15	5.33/6.98	4.01/4.46	485
GSAD	27.69/24.85	0.92/0.87	0.88/0.85	0.09/0.15	5.36/7.78	4.14/4.92	485

Results depicted in Table 1 show the ability to preserve good quality in a cross-dataset scenario. Specifically, we observe that Dimma has a much smaller quality drop than other methods. For example, HEP [7], while obtaining astonishing results on the in-domain dataset, failed to achieve good results in the cross-domain setup.

Limited data. To examine the usefulness of our framework in real scenarios when we have only a few real image pairs, we collected a dataset that we called FewShot-Dark (FS-Dark). The collecting procedure was similar to LOL [1]—we took two photos of the same scene with different camera light exposures. FS-Dark consists of a total of 14 image pairs (6 training, 4 validation, and 4 testing) taken by a Samsung Galaxy M52 mobile phone. For a fair comparison, we trained our Dimma model, LLFlow [4], and SNR-Net [18]. We also validated the LLFlow trained on the LOL dataset.

The results are shown in

Table 3. Quantitative comparison for FS-Dark. The best results are in bold, and the arrows indicate whether the metric should be minimized or maximized. All of the models were trained on FS-Dark (except for LLFlow (LOL), which was trained on LOL), consisting of as few as 6 training pairs, with the same setup as for LOL (except for the smaller batch size due to the dataset size).

Method	PSNR ↑	SSIM ↑	RGB-SSIM ↑	LPIPS ↓	DeltaE ↓	NIQE ↓
SNR-Net	19.43	0.78	0.75	0.42	9.59	4.61
LLFlow	19.46	0.81	0.79	0.35	9.69	3.50
LLFlow (LOL)	20.77	0.87	0.86	0.19	7.91	2.74
Dimma (ours)	25.56	0.88	0.86	0.22	5.77	2.67

, and the visual comparison is presented in Figure 6. For more image examples, see Figure A2 in Appendix E. Compared to models trained on FS-Dark, our method achieves better results in terms of all metrics, which proves Dimma strongly outperforms other methods when only a few ground-truth pairs are available for training. Dimma also surpasses LLFlow trained on the LOL dataset in almost all metrics, which suggests that models trained on one data collection might not generalize well to the others.

Table 2. Runtime analysis of VE-LOL for selected methods performed in the Kaggle P100 environment. We measure average processing time of a single image in seconds.

Method	Time [s]
LLFlow	0.438 ± 0.168
SNR-Net	0.065 ± 0.004
EnlightenGAN	0.028 ± 0.041
Dimma (ours)	0.026 ± 0.051
HEP	0.009 ± 0.046
RUAS	0.009 ± 0.055
Zero-DCE	0.004 ± 0.044

Table 2. Runtime analysis of VE-LOL for selected methods performed in the Kaggle P100 environment. We measure average processing time of a single image in seconds.

Method	Time [s]
LLFlow	0.438 ± 0.168
SNR-Net	0.065 ± 0.004
EnlightenGAN	0.028 ± 0.041
Dimma (ours)	0.026 ± 0.051
HEP	0.009 ± 0.046
RUAS	0.009 ± 0.055
Zero-DCE	0.004 ± 0.044

Light conditioning. To compare our light conditioning mechanism, we used the SICE [34] dataset consisting of image sequences with different lightness. Since most low-light image enhancement methods (e.g., [2,3,4,23]) are capable of lightness conditioning, we created the visual comparison of generated images with different conditioning values. Thanks to the lightening conditioning mechanism of UNet, we achieved remarkable precision in generating images with a certain level of lightening. The qualitative results, placed in Appendix E, emphasize the superiority of Dimma in terms of light conditioning.

Ablation study. To further establish the efficacy of our approach, we compared the results of semi-supervised training using Dimma with training UNet without the dimming module. The intention behind these experiments was to validate the rationality of our dimming module with MDN on the extremely small dataset. Using the dimming module exposes UNet to more natural images than training only on paired ones, which increases the model’s knowledge of the natural-light photos’ distribution. However, this comes at the cost of using synthetic inputs, which may not perfectly represent the true distribution of dark images. We also showed that using a deterministic convolutional network with the same architecture as the backbone of the MDN does not produce results as appealing as our MDN-based dimming module does.

Our ablation study examines the usefulness of the learnable dimming module. We compare it to a more naive approach of training only the UNet on the very same few pairs.

Table 4. Ablation study on the LOL dataset. The best results are in bold. Each experiment was conducted three times with different training samples. The files that were used for training are listed in Table A1 in Appendix B.

# Pairs	3	5	8
Supervised (w/o dimming module)
PSNR	21.05 ± 4.16	21.62 ± 4.19	22.18 ± 4.40
SSIM	0.81 ± 0.09	0.82 ± 0.09	0.83 ± 0.09
LPIPS	0.27 ± 0.09	0.25 ± 0.09	0.23 ± 0.08
Semi-supervised with deterministic dimming
PSNR	22.93 ± 2.31	24.07 ± 2.41	24.26 ± 2.40
SSIM	0.78 ± 0.06	0.78 ± 0.06	0.79 ± 0.06
LPIPS	0.32 ± 0.08	0.31 ± 0.08	0.30 ± 0.08
Semi-supervised with the MDN
PSNR	23.24 ± 2.25	23.83 ± 2.26	24.47 ± 2.22
SSIM	0.78 ± 0.06	0.79 ± 0.06	0.79 ± 0.05
LPIPS	0.31 ± 0.08	0.30 ± 0.08	0.29 ± 0.07
Semi-supervised with the MDN & few real pairs (Dimma)
PSNR	24.44 ± 0.76	24.93 ± 0.80	25.21 ± 0.36
SSIM	0.84 ± 0.01	0.85 ± 0.01	0.86 ± 0.01
LPIPS	0.23 ± 0.02	0.22 ± 0.02	0.20 ± 0.01

shows the rationality of the Dimma pipeline. In Figure A1 in Appendix D, we show the visual comparison of the color map generated by the MDN and the deterministic neural network. The comparison shows that the deterministic approach always generates the expected value of the color map pixels and cannot synthesize any variation due to the lack of stochasticity. The MDN, on the other hand, employs a mixture of Gaussians, which introduces randomness and creates distorted color maps that better align with the color maps of real-world dark images.

To further study our semi-supervised setup, we examined different probabilities p of choosing the real image pair during brightening module training. Higher p values move our model closer to the standard supervised method, which we proved to be less effective in the previous ablations. On the other hand, Dimma with small p values relies strongly on the data generated by the MDN and therefore can overfit to synthetic pairs. We examined three different values of p, and the results are shown in

Table 5. Ablation studies on the probability of passing the real image pair instead of the synthetic pair produced by the dimming module. The real pairs are the same as those that were used to train the dimming module. Experiments were conducted on the LOL dataset. We used PSNR and DeltaE as metrics, as they vary the most for different values of p. The best results are in bold, and the arrows indicate whether the metric should be minimized or maximized.

# Pairs	p	PSNR ↑	DeltaE ↓
	0.50	24.26	9.22
3 pairs	0.75	24.44	7.91
	0.90	24.21	8.00
	0.50	24.95	9.19
5 pairs	0.75	24.93	7.64
	0.90	24.85	7.67
	0.50	25.20	8.95
8 pairs	0.75	25.21	7.23
	0.90	25.13	7.27

. Based on this study, we chose the value of $p=0.75$.

5. Conclusions

In this paper, we propose a novel semi-supervised learning approach for low-light image enhancement. We address the problem of an insufficient number of large datasets containing image pairs for various cameras by utilizing only a few real image pairs, which can be easily captured and prepared. Based on these pairs, our model aims to mimic the characteristics of a specific camera in low-light conditions observed in the data. With this approach, we can effectively train existing models to restore dark images captured by that particular camera. Furthermore, our approach allows for the grading of the dimming factor during training, enabling the model to adapt to different light conditions and generate images with any desired illumination. As a result, our method significantly outperforms existing unsupervised models and achieves competitive results compared to fully supervised approaches.

For the FS-Dark dataset with limited training pairs, Dimma is superior to supervised methods [4,18]. This demonstrates Dimma’s ability to make better use of limited training samples. Additionally, Dimma significantly outperforms LLFlow [4] trained on the full LOL dataset in PSNR and DeltaE metrics, and it performs comparably in other metrics. This suggests that large, general, low-light image enhancement datasets are insufficient for different domains, such as those taken by different cameras or having different photo characteristics, highlighting many potential practical uses for Dimma.

Limitations. The main limitation of Dimma is the need to train the UNet backbone using a dimming module calibrated for particular camera settings. As a result, training must start from scratch for each new dataset. Secondly, our method achieves good results using only a few paired images, which makes the data-gathering process easier and cheaper but still necessary. Dimma cannot work without paired photos, which are much easier to collect.

As a potential future work, having a more general backbone and changing camera-specific modules on top for different datasets would enable few-shot fine-tuning, limiting the time and resources needed. The second drawback of Dimma highlights that the development of fully unsupervised methods that are on par with supervised models is the next crucial step in low-light image enhancement. A potential improvement in this direction could be replacing the MDN model within the dimming module with conditional GAN models for reflectance distortion, making the training process fully unsupervised, similar to [6].