A Lightweight Privacy-Preserving System for the Security of Remote Sensing Images on IoT

Abstract

The acquisition of massive remote sensing data makes it possible to deeply fuse remote sensing and artificial intelligence (AI). The mobility and cost advantages of new sensing platforms in the Internet of Things (IoT) make them ideal for continuous deployment rather than traditional airborne platforms. However, remote sensing devices are vulnerable to malicious attacks and privacy leaks when sharing data due to the complex architecture and heterogeneity of IoT and the lack of a unified security protection mechanism. Traditional protection methods based on public-key encryption require not only complex operations but also energy consumption, which poses new challenges for resources-limited IoT. The objective of this paper was to propose a lightweight privacy-preserving system for the security of remote-sensing images based on visual cryptography. This stacking-to-see feature of visual cryptography enables the efficient encryption of big data such as high-resolution and multi-scale remote sensing images in resource-constrained IoT. To alleviate image quality degradation in visual cryptography, we combined denoising neural networks to extract high-quality images from encrypted datasets, thus improving the recognition accuracy of loss datasets. We conducted extensive experiments, and the results verify the effectiveness of the proposed method in terms of privacy protection and classification accuracy.

1. Introduction

With the rapid development of mobile communication and information technologies, the development of IoT has sprung up and has been widely valued in recent years [1]. Through the integration of the three-tier architecture of the perception layer, the network layer, and the application layer with existing technology, the IoT has shown potential in fields such as smart homes, healthcare, autonomous driving, and digital earth [2,3,4,5]. As a technology wave and an industrial revolution in the current era, the construction of IoT relies on the technical support provided by remote sensing, which can perceive space, sky, earth, and sea from multiscale perspectives, and enrich the data sources and dimensions of the perception layer of the IoT. Remote sensing is a discipline that perceives and recognizes spatiotemporal information. Due to the vital role that remote sensing plays in earth research, geological surveying, disaster prediction, and military reconnaissance, it has become an important strategic resource of the country [6].

In recent years, we have gone beyond understanding traditional remote sensing technologies, such as aerial mapping, geographic information systems, and earth observation. The development of the IoT has allowed for a stream of remote sensing data for a long time. With the arrival of new data acquisition platforms, interconnected devices on these converged platforms can continuously transmit data The development of this integration includes the emergence of robot and UAV (unmanned aerial vehicle) platforms [7,8], and spectral sensors can be installed on smaller platforms. The mobility and cost advantages of these new platforms make them ideal for continuous deployment rather than more traditional airborne platforms. Lighter and cheaper sensors make this fusion even better [9]. The massive amount of remote sensing data acquired makes the deep fusion of remote sensing and AI possible. The emergence of AI enables remote sensing through the whole link of massive heterogeneous data, from processing and analysis to sharing. It shortens the interpretation cycle of remote sensing images and promotes the reform of the remote sensing data service mode, thus creating a new era of remote sensing.

To prevent the leakage of sensitive information such as coastal island topography, watercolor, and surface ships in remote sensing images, it is necessary to encrypt and store remote sensing images before uploading them to an open platform [10,11]. Due to the complex architecture of the IoT and the lack of a unified security protection mechanism, devices in the IoT are vulnerable to malicious attacks and privacy leaks when sharing remote sensing data [12]. The development of the IoT itself is restricted due to resource limitations and vulnerability to security attacks [13], which poses new challenges to the information security of remote-sensing images.

Traditional data protection methods based on public-key encryption require not only complex operations but also energy consumption [14]. The widespread use of IoT has exacerbated society′s concerns about the privacy leakage of remote sensing information. The sensor layer security technology is aimed mainly at threats faced by sensors with limited resources, small size, and low power consumption, which cannot use traditional security technologies that consume large resources to protect devices from eavesdropping [15]. Although different types of data protection algorithms have been proposed, most of them are aimed at text or digital, which is different from sensing images in information quantity and scale [16]. It is often unnecessary to encrypt all pixels in order to securely transmit an image [17]. Therefore, the traditional HTTPS and TLS (transport layer security) protocols that transmit data on the Internet are no longer applicable to resource-constrained IoT nodes for encryption of super-resolution and multi-scale remote sensing images. Developing trustable artificial intelligence technology and improving its privacy has become an important direction in technological governance.

To address the above issues, we propose a lightweight privacy-preserving framework for remote sensing data in IoT which securely transmits massive remote sensing images in plaintext in an open environment using the secret sharing feature in VC (visual cryptography). Through simple Boolean operations, this framework eliminates the complex computation and key dependency of traditional encryption methods, making it possible to securely transmit remote sensing images with computing-limited IoT. To mitigate noise interference in visual cryptography, which degrades recognition performance, we also adopt a denoising neural network to improve the recognition accuracy of encrypted images [18]. The main contributions of this paper are as follows.

(i) We propose a secure transmission algorithm that is independent of computing power through the stacking-to-see feature, which can efficiently transmit remote sensing images with super-high resolution and multiscale and alleviate the performance degradation of traditional public-key-based cryptography.

(ii) We propose a block encryption scheme to keep the size invariant of recovery images of the meaningful visual cryptography and to eliminate the preprocessing steps.

(iii) We combine a recognition model from large-scale datasets and denoising neural networks in order to further improve the recognition accuracy of remote sensing images towards lossy images recovered from visual cryptography.

The structure of the rest of the article is as follows. In Section 2, we will briefly review the security threats faced by remote sensing along with existing solutions and their shortcomings. Section 3 will introduce our lightweight privacy-preserving framework for the security of remote-sensing images in the IoT. In Section 4, we will evaluate the effectiveness of this proposed framework through extensive experiments on classic remote-sensing image datasets. We will conclude the paper and give future work in Section 5.

2. Related Work

Many remote sensing images, such as high-resolution satellite images related to public security, need to be kept highly confidential. One of the main aspects of remote sensing technology has been to ensure the security of big data, while deep learning needs to share these data in order to automatically learn their features. To address the issue, Wadii et al. [19] combined Paillier homomorphic encryption to improve the security of satellite images. The experimental results show that this scheme can realize privacy-preserving deep learning on ultra-high resolution remote sensing images. However, this scheme still relies on the traditional public-key system, and how to manage the key remains to be determined. To solve the problem of secure remote sensing image storage in an open cloud environment, Qi et al. [20] proposed an image-sharing scheme of weighting cloud service providers. The scheme achieves the secure and efficient storage of large-scale images through the Chinese Remainder Theorem.

The development of remote sensing has been bolstered by new data acquisition platforms in IoT. Researchers have conducted strategic planning and layout for the technology and application of IoT [21]. Yang et al. [22] developed a lightweight biometric privacy-preserving algorithm based on block logic operation to reduce the feature size of the neural networks model, thus saving memory and computing costs. To protect the privacy of images, it is necessary for the negotiation of secret keys and identity authentication. However, the public-key-based DTLS [23] handshake process is still a heavy security protocol for IoT nodes, and therefore, communication efficiency will be reduced when deploying the DTLS protocol on resource-constrained nodes.

Traditional methods including encryption, authentication, authorization, and signature ensure the confidentiality of the data by computational complexity, which is expensive for resource-constrained IoT terminals. Complex encryption operations also reduce the timeliness of IoT application response. Unlike the text-oriented secret-sharing mechanism, which divides the sum into two addends, VC [24,25] splits secret pixels into multiple subpixels. It is a subset of secret sharing that acts as a special solution for image security [26]. In the original VCS (VC scheme), one secret pixel was encrypted into multiple sub-pixels, resulting in pixel expansion. Based on the cover model, Zhang et al. [27] proposed a novel VCS with no size expansion that is nearly lossless through vertical expansion and contrast adjustment. However, this mechanism generates eight or more shares, which adds to the burden of encrypted image transmission.

EVCS (Extended VCS) [28] uses meaningful cover images instead of noisy sharing and maintains the VC threshold feature in order to avoid eavesdropping attention. However, it still suffers from pixel-expansion problems. Lee et al. [29] roposed a new general-purpose access structure generation algorithm in order to eliminate the expansion of pixels in traditional VC-based approaches. The scheme first uses a simulated annealing algorithm to generate a meaningless stockpile image, and then the black pixels in the source image are evenly distributed across the stock using a sampling algorithm. As multiple optimization methods are used simultaneously, the encryption of this scheme is more complicated than traditional VCS, and the restored image still reveals part of the image information. Wu et al. [30] combined integer linear programming and XOR operations to optimize the quality of recovery images and extend the mechanism to construct a $(k,n$)-EVCS encryption matrix. However, this scheme still cannot eliminate pixel expansion. It is necessary to keep the encrypted and decrypted images, since machine learning algorithms require a uniform data size of the probe images.

It is the premise of classification performance by reducing various noises introduced into the imaging system and obtaining high-quality remote sensing data. Due to sensor interference, periodic noise may appear in acquired remote sensing images. These noises may be generated in transmission and quantization. Image denoising can be divided into three categories: filtering-based methods, model-based methods, and learning-based methods. Learning-based methods focus on learning the potential mapping from noisy images to clean images. In recent years, depth networks have become the main methods because they have obtained more promising denoising results than methods based on filtering, model, and traditional learning [31].

The residual depth shrinkage network [32] can detect these noises with the help of an attention mechanism, which will improve the accuracy of image classification. DNCNN [18] focuses on denoising Gauss noise. FFDnet [33] emphasizes generalizing Gauss noise to more complex real noise. The network can be applied to images with different noise levels by entering a noise level map. CBDnet [34] synthesizes signal-independent noise and real noise images to improve the generalizability of the denoising network, enhancing the denoising performance. To ensure the security of image transmission, VCS inevitably mixes with noise. Therefore, we can adopt learning-based denoising methods to reduce noise interference, which in turn improves the classification performance of encrypted images.

3. Methods

The main ideas behind the lightweight privacy-preserving framework are (i) proposing an improved EVCS to efficiently and securely transmit images and (ii) improving the recognition accuracy of encrypted remote-sensing images by denoising neural networks. As shown in Figure 1, we can divide the framework into two parts. The first part is the secret sharing of remote-sensing images based on visual cryptography. We propose a lightweight secure transmission mechanism for large-capacity data such as images by the stacking-to-see feature of VC, which can efficiently transmit remote sensing images with super-high resolution and multiscale and alleviate the performance degradation of traditional public-key-based cryptography. The second part is image de-noising. To meet the security requirements of VC without a computing device, the quality of the decrypted images will be degraded. We combined a recognition model from large-scale datasets and denoising neural networks to further improve the recognition accuracy of remote sensing images toward lossy images recovered from VC. We will first introduce the improved EVCS.

3.1. An Improved Extended VCS

In order to overcome the limitations of traditional visual cryptography on color images, we used halftone technology to convert images of different modes into black-and-white binary images. Images are available in three modes: color, grayscale, and black-and-white. Most images are stored and displayed in a color mode so that the detail is retained. Due to the popularity of the error diffusion algorithm (ED) [35], images can also be stored in black and white, which not only occupies 1/5th of the storage space but also has a higher similarity to the original image [36].

ED is a commonly used halftone technique that quantizes the pixel of a grayscale image in order to generate a binary output. By spreading the quantization error to adjacent pixels according to the kernel, ED can compensate for the threshold loss. The low-pass filtering characteristics of the human eye make it possible to simulate the visual effect of continuous-tone imagery from a black-and-white binary image. A gradient-like binary image can be seen as a distribution of pixels in a two-dimensional space. The main use of ED is to convert a multi-level image into an image that can be printed with monochrome ink.

Let $p$ be a pixel on the 256-level grayscale image whose gray value is 120. If we want to convert $p$ to 16 levels of gray, the easiest way is to divide each pixel by 16, then the truncated value is $\lfloor 120/16\rfloor =7$, which has an error of 0.5. As shown in Figure 2, the $P_{x,y}$ is the original pixel in the $x$ row and $y$ column, the $e_{x,y}$ is the truncated error, and the $K_{x,y}$ is the kernel matrix. To spread the quantization error, 0.5 is distributed to the right, bottom right, and bottom points in a 7:3:5:1 ratio, respectively. Equation (1) is the widely used kernel for $p$ in the ED process, where $p_{old}$ is the pixel in the current row that has been processed.

$$K_{x,y}=\left[\begin{array}{c}\begin{array}{ccc}{p}_{old}& p& 7/16\end{array}\\ \begin{array}{ccc}3/16& 5/16& 1/16\end{array}\end{array}\right]$$

(1)

The ED algorithm can convert a grayscale pixel ($G_b\left(x,y\right)$) to a halftoned pixel ($H_b\left(x,y\right)$). The grayscale image $G$ is divided into non-overlapping regions $G_b$ whose size is the same as the threshold matrix $T$. By comparing the gray level with the corresponding $T\left(x,y\right)$, $G\left(x,y\right)$ will be directly truncated to the maximum ($g_{max}$) or minimum ($g_{min}$) gray level.

$$H_b\left(x,y\right)=\{\begin{array}{c}{g}_{max},if G_b\left(x,y\right)<T\left(x,y\right);\\ g_{min},if G_b\left(x,y\right)\ge T\left(x,y\right)\end{array}$$

(2)

Table 1. The (2,2)-EVCS encryption and decryption example for a white pixel.

Secret Pixel
	$S_0^{11}$	$S_0^{10}$	$S_0^{01}$	$S_0^{00}$
$S_{\mathrm{c}1}$
$S_{\mathrm{c}2}$
Decrypted Pixel

shows the (2,2)-EVCS encryption and decryption example for a white pixel. We used 0 to denote white pixels and 1 to denote a black pixel, which is inconsistent with the display mode, but consistent with the printing mode of images. Since black images or texts are usually printed on a white background, by splitting a secret binary image into $n$ shared images, VC can generate encrypted images that can be decrypted directly by human eyes without any digital devices.

In EVCS, in addition to a secret image, there are two meaningful cover images. The pixel in the corresponding position of the first cover image ($C1$) will be white (0), and the pixel in the second cover image ($C2$) will be black (1); the candidate encryption matrix is as follows:

$$S_s^{c1c2}=S_0^{01}$$

(3)

which is shown in the third column of Table 1. The black and white pixels in the encrypted blocks are no longer equal in contrast. The distinguishable encryption makes it possible to present information on shares. The number of black pixels in the encrypted white blocks is 2:

$$S_s^{c1c2}=S_0^{c1c2},OR\left(p_{c1},p_{c2}\right)=0$$

(4)

The black pixels in the black blocks are 3:

$$S_s^{c1c2}=S_0^{c1c2},AND\left(p_{c1},p_{c2}\right)=1$$

(5)

Therefore, the contrast of the black blocks is higher than that of the white, taking advantage of the low-pass filtering properties of human eyes. The encrypted subpixels $p_{c1}$ and $p_{c2}$ are still meaningful but are four times the size of the original.

$$S_p^{c1c2}=\left[\begin{array}{ccc}{s}_{11}& s_{12}& \begin{array}{cc}{s}_{13}& s_{14}\end{array}\\ s_{21}& s_{23}& \begin{array}{cc}{s}_{23}& s_{24}\end{array}\end{array}\right]$$

(6)

$$\{\begin{array}{c}\sum s_{1i}=={\tau }_c\left(p_{c1}\right)\\ \begin{array}{c}\sum s_{2i}=={\tau }_c\left(p_{c2}\right)\\ OR\left(s_{1i},s_{2i}\right)=={\tau }_c\left(p_s\right),i<4\end{array}\end{array}$$

(7)

Note that although the encryption matric $S_0^{00}$ for $C2$ is inconsistent with other color blocks, the left white and right black blocks also appear in the $S_1^{10}$ for $C2$. Sharing blocks corresponding to a secret pixel may all appear on the other side, and the attacker cannot infer the original secret pixel through a single share.

For a (2,2) VCS, if the Hamming distance $H\left(V\right)$ is greater than $d$, the human eye will interpret the pixel as white, otherwise as black, and the matrix $V$ is a combination of encryption matrices of the $q$ line. $a$ is the contrast between black and white pixels and $m$ is the expanded size of the pixels.

$$\{\begin{array}{c}H\left(V\right)<d-am,for S_0\in C1\\ H\left(V\right)>d,for{S}_1\in C2\\ V=\left\{S_i^0,\dots ,S_i^q\right\},0\le i<2, 0\le q<k\end{array}$$

(8)

Similar to the traditional VCS, EVCS encrypts an image pixel-by-pixel, resulting in pixel expansion. We can deal with equal-sized secret blocks once to issue the limitation. If the color block is directly preprocessed by ED, the generated color blocks may contain black pixels ranging from 0 to 4. However, as shown in Table 1, only two or three black patches are allowed in the pre-processed cover images in the original EVCS. Therefore, in case the number of black pixels of sharing blocks is 0,1, or 4, the existing general access structures cannot be reused. In addition to the binary images, we can also set threshold functions to generate tertiary images or quaternary images as follows:

$$c_b=L\lfloor \mathrm{T}-\tau (\sum B_g))\times l_L/(l_b+1)\rfloor$$

(9)

where $L$ is the allowable gray level, which generally takes the value of $\left[2,3\right]$ or $\left[3,4\right]$ for EVCS; $\tau$ is the threshold function, which takes the value of 4 for the (2,2)-EVCS; and $B_g$ is the gray value of the color block. $l_L$ is the number of allowable gray values, which takes the value of 2, and $l_b$ is the number of pixels in the color block, which takes the value of 4.

Equation (9) ensures that the number of black pixels in the pre-processed color blocks of the shares will not exceed that of the corresponding pre-processed secret blocks after superposition, making block-by-block encryption possible.

$$\{\begin{array}{c}{\mathrm{c}}_b^{B_s}\left(i,j\right)\le \min \left(c_b^{B_{c1}}\left(i,j\right), c_b^{B_{c2}}\left(i,j\right)\right), 0\le i<M/m,0\le j\le N/n\\ m\times n\ge c_b^{S_{c1}}+c_b^{S_{c2}}\end{array}$$

(10)

where ${\mathrm{c}}_b^{B_s}\left(i,j\right)$ is the count of the $i$ row and $j$ column of the secret block $B_s$; $c_b^{B_{c1}}\left(i,j\right)$ and $c_b^{B_{c2}}\left(i,j\right)$ is the count of the encode cover blocks $c1$ and $c2$, whose size is equal with $B_s$. $M$ and $N$ are the sizes of the secret image and cover images. $m$ and $n$ are the sizes of encryption blocks, which are often $2\times 2$.

As shown in Algorithm 1, for the original (2,2)-EVCS, the allowable gray levels in the encoding cover images are 2 and 3, and that in the decrypted image is 3 or 4:

$$\{\begin{array}{c}{g}_p^{B_s}\in \left[3,4\right],\\ g_p^{B_{ck}}\in \left[2,3\right],k\in \left[1,2\right]\end{array}$$

(11)

To meet the security of VC, although there are no all-white blocks for decryption, as long as the contrast between a black block and a white block is different, the human eye can still reveal meaningful information from decrypted images.

Algorithm 1: The encoding process of the size-invariant EVCS.

3.2. High-Accuracy Recognition Neural Network for Encrypted Datasets

Remote sensing needs AI and big data technologies to intelligently integrate spatial–temporal data, which elimates errors and redundant information with the complex format to reveal the real value hiding under the data. However, remote sensing images are characterized by arbitrary orientation, scale diversity, geometric deformation, and dense arrangement of target remote sensing images [37]. Therefore, an improvement based on deep learning is required to combine the features of remote-sensing images [38].

To ensure the security of image transmission, VCS inevitably mixes with noise. On the other hand, remote sensing images are often polluted by noise in the process of the acquisition, transmission, reception, and output of remote sensing images. Therefore, remote sensing images appear blurry in details such as edge texture, leading to image quality degradation, inability to reflect the features of ground objects truly and objectively, and increased difficulty of remote sensing image post-processing and analysis.

We can adopt learning-based denoising methods in order to reduce noise interference, which in turn improves the classification performance of encrypted images. The cores of deep learning include computing power, algorithms, and big data [39]. Deep learning uses supervised learning to learn the essential characteristics of the data through a large number of samples. The trained model can predict and judge the unknown data accordingly. Although we can design structure and loss function neural networks for specific datasets and fine-tune the model and hyperparameters in order to obtain the best recognition performance, this method is contrary to general AI, and the performance of the model will rapidly decline when applied to new tasks or datasets [40]. Additionally, there is a cost in terms of computational power to train models repeatedly for different datasets.

The merit of transfer learning is to enhance learning toward the target task by reusing knowledge in the existing and general tasks [41]. The learning task $D=\left\{X,P\left(X\right)\right\}$ consists of feature space $X$ and probability distribution $P\left(X\right)$, the sample data consists of pairs $\left\{x_i, y_i\right\}$, where $x_i\in X, y_i\in Y$ and the objective function $f$ should be learned from the sample in order to predict the corresponding labels of new instances. Given the source domain $D_S$ with corresponding source task $T_S$ and the target domain $D_T$ with the corresponding target task $T_T$, transfer learning aims to transfer the relevant knowledge contained in $D_S$ and $T_S$ in order to improve the performance of the target prediction function $f_T$ in target task $T_T$ and target domain $D_T$.

As shown in Algorithm 2, we carried out joint training by combining images based on signal-independent noise and noise processed inside the clean images and real noise images in order to improve the generalization ability of the denoising network. The high-accuracy recognition network guides the denoising process of noise images recovering from VC by learning the Gaussian mixture before external clean images, after learning the external Gaussian mixture model, and then using the low-rank estimation process to restore clean images.

Algorithm 2: A high-accuracy recognition neural network for encrypted datasets.

4. Experiments and Discussion

In this section, we will carry out extensive experiments to evaluate the effectiveness of the proposed method.

As shown in Figure 3, we show the encryption and decryption of the sensing images in land-use, biometric, and traffic signs images. The size of all pictures is $256\times 256$. The first three rows are three randomly selected samples from datasets. When an image is encrypted, the other two images are treated as cover images. The last three rows are the corresponding decrypted images. It can be seen that the size of the image remains the same before and after encryption, and the decrypted image is darker because we limit the appearance of some color blocks, but we can still see the features of the image from the decrypted image.

Figure 4 shows the encrypted transmission performance of our lightweight framework. Note that the y-axis is on a logarithmic scale. plain is the transmission time of the unencrypted image end-to-end, and TLS denotes the consumed encryption time of the traditional encryption channel. The data in the figure represent a hot start, that is, the initially established encryption channel will be reused to transmit the subsequent data. Therefore, the subsequent handshake and key distribution through the public key certificate system are not required, but the image is transmitted using symmetric encryption. As our transmission scheme only carries out simple Boolean operations, its encryption transmission performance is equivalent to that of normal images, about 8 times faster than TLS.

Furthermore, we take the MSE (mean square error), PSNR (peak signal noise ratio), and SSIM (structural similarity index measure) metrics [42] to quantitatively test the quality of the restored images. $S\left(i,j\right)$ and $R\left(i,j\right)$ are the original secret pixel and decrypted pixels, respectively. $MA{X}_I$ is the highest gray value. ${\mu }_x$ and ${\mu }_y$ are the gray means of images $x$ and $y$. $C_1$ and $C_2$ are used to prevent division by zero. The reference image is the first two columns of Figure 3.

$$MSE=\frac{1}{M\times N}{\displaystyle \sum }_0^M{\displaystyle \sum }_0^N\parallel S\left(i,j\right)-R\left(i,j\right)\parallel {}^2$$

(12)

$$PSNR\left(S,R\right)=10\times {\log }_{10}\left(\frac{MA{X}_I^2\times m\times n}{{{\displaystyle \sum }}_{i=0}^{n-1}{{\displaystyle \sum }}_{j=0}^{m-1}\parallel S\left(i,j\right)-R\left(i,j\right)\parallel {}^2}\right)$$

(13)

$$SSIM\left(x,y\right)=\frac{\left(2{\mu }_x{\mu }_y+C_1\right)\left(2{\sigma }_{xy}+C_2\right)}{\left({\mu }_x^2+{\mu }_y^2+C_1\right)\left({\sigma }_x^2+{\sigma }_y^2+C_2\right)}$$

(14)

As shown in

Table 2. The comparisons of image quality from the original halftone, the proposed EVCS, Wu’s and Ren’s methods.

PSNR (%)|MSE								SSIM (%)
Halftone		Ours		Wu [30]		Ren [44]		Halftone	Ours	Wu [30]	Ren [44]
4.75	170.83	4.89	165.41	4.75	170.83	4.74	170.83	7.81	7.32	2.34	1.01
4.81	168.49	4.76	170.44	4.71	172.41	4.79	172.41	3.83	3.07	1.26	0.13
4.93	163.90	5.00	161.28	4.98	162.02	4.74	162.02	8.35	3.47	1.51	1.17
4.76	170.44	5.13	156.52	4.70	172.81	4.66	172.81	2.12	1.83	0.08	0.64
5.15	155.80	4.72	172.02	4.79	169.27	4.74	169.27	12.78	11.79	3.11	0.94
4.82	168.10	4.92	164.27	4.54	179.30	4.72	179.30	6.04	5.88	1.92	1.12
4.71	172.41	4.76	170.44	4.67	174.01	4.64	174.01	6.12	5.92	1.13	0.33
4.76	170.44	5.16	155.44	4.82	168.10	4.62	168.10	7.13	6.12	2.23	1.12
5.12	156.88	4.56	178.47	4.92	164.27	4.74	164.27	6.90	6.32	2.93	0.34
4.82	168.10	5.01	160.91	5.04	159.80	4.67	159.80	6.91	5.87	2.94	0.49

, because there are so many images in the dataset, we randomly selected 10 images from the UCMercd dataset [43]. UCMerced is a classic remote sensing scene recognition dataset used to classify land-use scenarios in urban areas. The spatial resolution of the dataset is about 0.3 m, and the image scale is $256\times 256$, including 21 types of scenes, 100 images of each type, and a total of 2100 images. When encrypting an image, we select 2 of the total 9 images as cover images. The average image quality is obtained after repeating this 100 times. The PSNR values of images generated from the classical ED algorithm, our, Wu′s, and Ren’s methods are around 4.8, and MSE is about 170. Although MSE and PSNR are easier to calculate, they cannot distinguish black-and-white binary images well. The second column of the table is the SSIM values of the corresponding images. It can be seen that the halftoned image has the highest SSIM value. Although our method is lower than that of halftone images, the diversity is within a reasonable range, while the SSIM value of Wu′s method is far lower than our halftoned method. Ren′s method has the worst performance. They know how to select that available encryption matrix through a simple search, and they have only two available encryption matrices, which is fewer than ours. The degradation comes from the fact that, although the probabilistic method can directly reuse the existing EVCS, the underlying VCS schemes also limit the highest contrast they can achieve. The probabilistic method can only take one pixel from the entire row matrix at a time. Other permutations will be discarded. For (2,2)-EVCS, only 1/5 of the pixel information is retained. However, our method only discards blocks with specific permutations. For our (2,2)-EVCS, that is, 2/5, a decryption block may contain 3 or 4 black pixels, which improves the visual detail appearing in recovery images. So, we can conclude that the more gray levels that are discarded, the worse the image quality.

Figure 5 shows the recognition performance (ROC) of different models in the classic UCMercd dataset. It can be seen that we cannot directly reuse the existing depth learning model to identify the encrypted remote sensing image with noise, but by introducing the noise removal model based on a neural network, we can improve the recognition accuracy from 0.92 to 0.96, which is almost the same as the unencrypted data set.

We adopt ResNet [45] as a backbone to evaluate the recognition performance of the UCMercd dataset and then append a fully connected layer that adapts to the categories of remote sensing datasets as classifiers. Normal in Figure 5 indicates the best performance with an AUC (area under curve) of 0.97. If we adjust the parameters and optimize the network structure, we can obtain better results. Encrypted denotes the recognition performance on encrypted datasets. As VC introduces global noise, the recognition performance drops significantly by 0.05. However, since the human eye can still recognize the information in the blurred image, the recognition network based on deep learning can still achieve an AUC of 0.92. The BM3D [46] (block-matching and 3D filtering) method filters images by finding similar blocks. Although BM3D adopts the traditional statistical denoising method, recognition performance improves by 0.02 after denoising. If we denoise the encrypted images based on CBDnet prior to classification by the finetuned ResNet, an AUC of 0.96 can be obtained, which is only 0.01 lower than that of a normal dataset. We can conclude that de-noising preprocesses before image classification can significantly improve the recognition performance of lossy encrypted datasets.

Figure 6 shows the ROC on the RSSCN7 [47] dataset. Since remote sensing datasets often have different resolutions, we test the recognition performance of the encrypted datasets on RSSCN7 with multiresolution. The four scales are 1:700, 1:1300, 1:2600, and 1:5200, respectively. The size of images in the data set is $400\times 400$. It can be seen that the resolution has little impact on the recognition performance due to the import of denoise networks, and the AUC is about 0.96. We can conclude that, although our lightweight encryption scheme introduces noise, it can still maintain the recognition performance on the encrypted multiscale remote sensing dataset through the denoising network.

The contrast of the recovered secret image may impact the recognition performance, since different thresholds would have different permutations.

Table 3. Comparisons of the recognition performance for different numbers of images shared in the RSSCN7 dataset.

Schemes	(3,3)-EVCS	(4,4)-EVCS	(6,6)-EVCS	(8,8)-EVCS
Accuracy	0.94	0.91	0.83	0.64
Precision	0.92	0.92	0.85	0.62
Recall	0.93	0.92	0.84	0.62
F1-score	0.92	0.93	0.84	0.61

shows that, with an increase in the number of images shared, the recognition performance will decrease, especially when the (8,8)-EVCS is adopted, and the recognition performance drops significantly to 0.64. This is because the recognition model that is transferred from large-scale datasets has a strong generalization; however, the encryption matrix of EVCS originated from VCS, whose contrast is inversely proportional to the number of shares, so the recognition accuracy rate decreases with increasing shares until it exceeds the generalization of the model. As the distribution of each class in the RSSCN7 dataset is relatively balanced, the trend of other indicators, such as precision, recall, and F1-score, is consistent with the accuracy.

5. Conclusions

In recent years, the IoT and AI have been integrated to promote the interconnection between information and physical space. Remote sensing image classification has been the cornerstone of remote sensing applications since it is essential for the conversion of remote sensing images into understandable geographic data. We have gone beyond understanding traditional remote sensing technologies. Emerging big data technology is one of the key factors for the success of artificial intelligence algorithms based on neural networks. To prevent the leakage of sensitive information such as coastal island topography, watercolor, and surface ships in remote sensing images, it is necessary to encrypt and store remote sensing images before uploading them to an open platform. However, the traditional public-key-based encryption methods remain the limitation of key dependence and computational complexity.

Given the importance of remote-sensing images to people′s livelihood and national defense forces, a lightweight privacy-preserving system for the security of remote-sensing images based on visual cryptography is designed in this paper. Through simple Boolean operations, the framework eliminates the complex computation and key dependency of traditional encryption, making it possible to securely transmit big data, such as high-resolution and multiscale remote sensing images in resource-constrained IoT. To alleviate noise interference in visual cryptography degrades data recognition performance, we further adopt a denoising neural network to improve the recognition accuracy of encrypted images. The experimental results of remote sensing image datasets show that the proposed framework provides privacy and maintains the high accuracy of remote sensing image recognition.

In future work, we will design secure transmission methods based on the scale, multiresolution and content characteristics of remote sensing images. Although we cannot eliminate the noise in visual cryptography, we alleviate this interference with the characteristics of remote-sensing images. In addition to the classical denoising network, we are also studying the method for denoising neural networks for encrypted datasets, which will further improve recognition performance. Last but not least, as a privacy protection framework, this paper focuses on the computational complexity of encryption. We will conduct more attack experiments to evaluate the robustness of the method.