*2.1. Overview of the Method*

In this work, a multi-spectral domain translation generation adversarial model is proposed for remote sensing images. Following the basic framework of image-conditional GANs [12], the model has an independent encoder E, a decoder G, and a discriminator D for each spectral domain. The difference is that the model assumes the existence of a shared latent domain, which make it possible to encode each spectral domain into that space and reconstruction of information from that space.

In the training process, the shared latent domain is constructed in two ways. First, the source domain spectral image and the target domain spectral image are encoded to the feature matrix with the same size. The training with L1 loss makes the encoded feature matrix consistent across spectral domains. Second, in within domain training, the source and target domains use their encoders and decoders to achieve image reconstruction from the feature matrix. In cross domain training, the feature matrices output from the source and target domain encoders are exchanged, and then the images are reconstructed following the above steps. The purpose of this step is to enable decoders in different spectral domains to obtain the information needed for their reconstruction from the shared latent domain.

During the test, there is no need to reload all the encoders and decoders. Only the combination of encoders for the source domain spectrum and the combination of decoders for the target domain need to be loaded. The feature matrix is generated by the encoder in the source domain, and then the spectral image is generated by the decoder in the target domain.

Since all encoding and decoding is based on the shared latent domain, the set of spectral domains of the model can be continuously expanded. When a new spectral domain is added, it is only necessary to ensure that the encoder of the new spectral domain can make the image output to the shared latent domain and the decoder can recover its own image from that space.

Meanwhile, the model can add additional physical property information to improve the simulation accuracy. For remote sensing imaging, the underlying surface and clouds are the main influencing factors of optical radiation transfer. Therefore, earth surface classification data *RGT* and cloud classification data *RCLT* are used as feature maps to form the boundary conditions of the scene.

### *2.2. Shared Latent Domain Assumption*

Let *xi* ∈ *χi* be the spectral images from spectral domain *χi*, and there are *N* spectral domains. Let *r* ∈ *R* be the condition information of image boundary condition *R*. Our goal is to estimate the conditional distribution *p xi xj*,*<sup>r</sup>* between domains *i* and *j* with a learned deterministic mapping function *p xj*→*<sup>i</sup> xj*,*<sup>r</sup>* and the joint distribution *p*(*<sup>x</sup>*1, *x*2, ··· , *xN*,*<sup>r</sup>*).

To translate from one spectral domain to multiple spectral domains, this study makes a fully shared latent space assumption [17,29]. It is assumed that each spectral image *xi* is generated from a latent code *s* ∈ *S* that is shared by all spectral domains and conditional information. Using the shared latent code domain as a bridge, spectral image *xi* can be synthesized by decoder *G*∗ *i* (*s*), and the joint probability distribution *s* can be obtained by encoder *E*∗ *i*(*xi*,*<sup>r</sup>*), so that *E*∗ *i*(*xi*,*<sup>r</sup>*) = *G*∗ *i*(*s*) −1 = *s*.
