Differential Privacy-Based Location Privacy Protection for Edge Computing Networks

Abstract

Mobile Edge Computing (MEC) has been widely applied in various Internet of Things (IoT) scenarios due to its advantages of low latency and low energy consumption. However, the offloading of tasks generated by terminal devices to edge servers inevitably raises privacy leakage concerns. Given the limited resources in MEC networks, this paper proposes a task scheduling strategy, named DQN-DP, to minimize location privacy leakage under the constraint of offloading costs. The strategy is based on a differential privacy location obfuscation probability density function. Theoretical analysis demonstrates that the probability density function employed in this study is valid and satisfies $ϵ$-differential privacy in terms of security. Numerical results indicate that, compared to existing baseline approaches, the proposed DQN-DP algorithm effectively balances privacy leakage and offloading cost. Specifically, DQN-DP reduces privacy leakage by approximately 20% relative to baseline approaches.

1. Introduction

The rapid development of the Internet of Things (IoT) and 5G technology has driven the integration of various smart applications with mobile devices, such as facial recognition and smart healthcare [1]. Mobile applications are becoming increasingly complex, reflecting a trend towards diversification. With users increasingly favoring lightweight and compact mobile devices, there has been a surge in new advanced applications requiring computation-intensive tasks [2]. However, the computational power, storage capacity, and network transmission capabilities of mobile devices are often limited, making it challenging to meet users’ Quality of Service (QoS) requirements. To efficiently address computation-intensive tasks on resource-constrained mobile devices, these devices offload a portion of the tasks to servers with abundant processing power. Consequently, cloud computing, centered around high-performance cloud servers, has emerged as a solution [3]. However, the physical or logical distance of cloud servers from users often results in longer transmission delays, failing to meet the latency requirements of time-sensitive tasks [4]. Thus, Mobile Edge Computing (MEC) technology has been developed to address these challenges [5]. By offloading computing tasks to resource-rich edge servers, MEC not only enhances QoS but also boosts the capabilities of mobile devices for applications with higher resource demands. Furthermore, the distributed structure of MEC servers prevents data congestion, resulting in significantly reduced transmission latency. Therefore, MEC is a promising paradigm that supports critical latency-sensitive services and a variety of IoT applications [6].

However, while MEC offers efficient computing services to users, it also poses significant risks to users’ privacy [7]. Malicious edge servers could infer users’ locations, offloading patterns, and other sensitive information by analyzing wireless channel data. Consequently, ensuring users’ privacy has become a critical concern in MEC deployments. In practical applications, to prevent the disclosure of privacy for sensor vehicles in vehicular networks, Xu et al. [8] proposed a personalized location privacy protection mechanism. In the context of smart cities, to safeguard the privacy of edge servers, Xu et al. [9] introduced a reputation-based service allocation method. Furthermore, an overemphasis on privacy protection may result in increased costs and latency, potentially diminishing the QoS for users. Thus, striking a balance between reducing the cost of task offloading and ensuring robust privacy protection remains a critical challenge.

To address the aforementioned challenges, this paper presents a novel DQN-DP algorithm for optimizing location privacy protection tasks based on differential privacy [10] and reinforcement learning [11,12]. The DQN-DP algorithm simultaneously provides location privacy protection while leveraging the DQN algorithm to make optimized offloading decisions. The main innovations of this paper include the following:

(1)

We propose a distance-obfuscating probability density function grounded in the principles of differential privacy theory. This function adeptly adjusts the level of privacy leakage by employing Kullback–Leibler divergence, while incorporating the ratio of offloading costs under both privacy protection and efficiency priority as optimization constraints. Our overarching objective is to minimize privacy leakage, thereby defining the obfuscation interval for users’ true locations.

(2)

Within the established obfuscation range, we design an offloading strategy aimed at minimizing offloading costs. The DQN algorithm serves as the foundational framework for learning offloading strategies to achieve optimized offloading decisions for enhanced service efficiency.

(3)

By demonstrating that the proposed DQN-DP algorithm satisfies $\epsilon$-differential privacy, we validate its efficacy in preserving user location privacy. Simulation results further corroborate the capability of the DQN-DP algorithm to render cost-effective offloading decisions.

The remainder of this paper is structured as follows: Section 2 reviews the current state of the relevant work, highlighting the strengths and limitations of existing studies. Section 3 provides a detailed description of the offloading model proposed in this paper. Subsequently, Section 4 presents our privacy-preserving model and further formulates the optimization problem addressed in this work. To solve this problem, we elaborate on the proposed optimized DQN-DP algorithm in Section 5. Finally, to demonstrate the reliability and efficiency of the proposed algorithm, we conduct simulations in Section 6 and offer a comprehensive conclusion in Section 7.

2. Related Works

In terms of resource allocation, several strategies have been proposed [13,14,15,16,17]. To provide satisfactory computational performance, Zhang et al. [13] introduced a Multi-Device Hybrid Actor–Critic (MD-Hybrid-AC) deep reinforcement learning algorithm for dynamic computational offloading. The MD-Hybrid-AC adopts a centralized training and decentralized execution framework. It learns coordinated decision making by developing a centralized critic that determines server selections, taking into account the continuous action strategies of all devices. To investigate the trade-off between energy consumption and latency, Zhang et al. [14] proposed an energy-aware offloading scheme that optimizes communication and computational resource allocation under limited energy and sensitive latency conditions. To address latency constraints and uncertain resource demands inherent in heterogeneous computational tasks within a dynamic multi-user MEC system, Zhou et al. [16] formulated the challenge as a Mixed-Integer Nonlinear Programming (MINLP) problem. They subsequently introduced a reinforcement learning approach, specifically leveraging value iteration and Q-learning, to ascertain the optimal joint strategy for computational offloading and resource allocation, aimed at minimizing overall energy consumption. In a multi-user MEC network with time-varying wireless channels and randomly arriving user task data, Bi et al. [17] aimed to design an online computational offloading algorithm to maximize network data processing capacity while adhering to long-term data queue stability and average power consumption constraints.

To safeguard privacy, various strategies have been proposed [18,19,20,21,22]. During the task offloading process, He et al. [18] introduced a privacy-aware task offloading scheduling algorithm using the Constrained Markov Decision Process (CMDP) based on location privacy and usage pattern privacy. He et al. [19] introduced a novel deep reinforcement learning algorithm that leverages additional energy collection to accelerate the learning of privacy-conscious offloading strategies for IoT devices. Nowadays, privacy concerns arising from data sharing are receiving increasing attention. Wang et al. [20] developed an algorithm that selects the optimal subset of public data to be released under differential privacy based on the perspective of non-interactive learning. To mitigate threats posed by untrusted third-party data collectors to users’ private information, Wang et al. [21] proposed a novel participant location protection method based on local differential privacy preferences. They implemented a chosen local differential privacy technique to obscure the locations of participants within designated regions. This involved utilizing discretized maps and density-based map segmentation. Subsequently, the obscured location data were transmitted to the collection server to safeguard the privacy of participant locations. Zhang et al. [22] introduced a fair contract signing scheme utilizing Ethereum smart contracts. This approach enables participants to execute contract signing procedures fairly on the blockchain, thereby eliminating the need for a central arbitrator.

However, while the aforementioned strategies may yield favorable outcomes, they do not integrate computational performance and privacy protection. To achieve an optimal solution through joint optimization, several strategies have been proposed [23,24,25,26,27]. In the training context of task offloading, Xu et al. [23] formalized the problem into a multi-objective optimization, considering load balance variance, average transmission time, and privacy entropy. They proposed a privacy-aware task offloading approach using the strength Pareto evolutionary algorithm to collaboratively enhance training performance and privacy protection. Liu et al. [24] introduced a privacy-preserving computation offloading scheme based on the whale optimization algorithm. They employed differential privacy techniques to obfuscate user location information, enabling task offloading decisions based on fuzzy distance. The task offloading problem was formulated as a convex optimization problem considering delay, energy consumption, and their weighted sum. To address security and cost issues in task transmission, Zhang et al. [25] proposed a task transmission strategy based on differential privacy and reinforcement learning. This strategy optimized the overhead associated with task offloading while safeguarding user privacy, by taking into account the resource allocation to select appropriate MEC servers for task offloading. Gao et al. [26] proposed a privacy-oriented task offloading approach, where privacy loss was formalized as the probability of a successful attack by an adversary with prior knowledge. While these approaches comprehensively consider the balance between computational performance and protection effectiveness, there are still shortcomings in terms of performance. Compared to the aforementioned studies, the DQN-DP approach is more comprehensive, as illustrated in

Table 1. Comparison of related works.

Property	[13]	[14]	[16]	[18]	[21]	[22]	[23]	[25]	DQN-DP
Latency	✓	✓	✓	✓	✓	✓	✓		✓
Energy consumption	✓	✓	✓	✓	✓	✓		✓	✓
Privacy leakage					✓		✓		✓
Location privacy				✓	✓	✓		✓	✓
Cost-effectiveness	✓	✓				✓	✓	✓	✓

✓: Indicates that the property is addressed in the corresponding work.

.

3. Task Offloading Model

The edge offloading model is illustrated in Figure 1. First, the edge server calculates the optimal location obfuscation range with the goal of minimizing privacy leakage. Then, the user generates an obfuscated address within this range and offloads tasks to the edge server using this obfuscated address, thereby preventing location privacy leakage during the task offloading process. In this chapter, we will introduce the communication model, the local computation model, and the edge computation model, respectively.

3.1. Communication Model

Users offload tasks to the edge server via wireless channels. We assume that the channel fading between users and MEC servers is independent and identically distributed. Therefore, at time t, the small-scale fading channel power gain from the user to the MEC server is denoted as $h\left(t\right)$. Consequently, the channel gain from the user’s mobile device to the MEC server is as follows:

$$H\left(t\right)=h\left(t\right)g_0{\left({\displaystyle \frac{l_0}{l}}\right)}^{\vartheta },$$

(1)

where $g_0$ denotes the path loss constant and $\vartheta$ donates the path loss exponent, with $\vartheta$ being the reference distance, typically one meter. Assuming the wireless channel bandwidth is B, the upstream link rate $r_i$ for user i when K users simultaneously require channel occupancy for offloading can be expressed as:

$$r_i=\left(B/K\right){log}_2\left(1+{\displaystyle \frac{{\tau }_{i}H\left(t\right)}{\left(B/K\right)N_0}}\right).$$

(2)

Here, ${\tau }_i$ represents the transmission rate of user i, while $N_0$ signifies the environmental noise.

3.2. Local Computing Model

We consider user i generating tasks of size ${\omega }_i={\omega }_i^{loc}+{\omega }_i^{mec}$. Here, ${\omega }_i^{loc}$ denotes the size of tasks processed locally, while ${\omega }_i^{mec}$ represents the size of tasks offloaded to the edge server. The local computation delay is expressed as:

$$T_i^{loc}={\displaystyle \frac{{\omega }_i^{loc}·\beta }{f_i^{loc}}},$$

(3)

where $f_i^{loc}$ is the processing frequency at the user i in CPU cycles and $\beta$ is the number of CPU cycles required by the user to process each byte. Defining $E_i^{loc}$ as the energy required for local calculations, it is expressed as [28]:

$$E_i^{loc}=\kappa {\left(f_i^{loc}\right)}^2{\omega }_i^{loc}\beta .$$

(4)

The energy consumption per CPU cycle is defined as $\epsilon =\kappa f^2$, where $\kappa$ denotes the CPU energy consumption factor and it is typically set to ${10}^{-25}$.

3.3. Edge Computing Model

The computation latency $T_i^{comp}$ of the MEC server is influenced by the allocation of computational resources to user i by the edge server. Mathematically, the computation latency of the edge server can be expressed as:

$$T_i^{comp}={\displaystyle \frac{{\omega }_i^{mec}·\beta }{f_i^{mec}}}.$$

(5)

Task data uplink communication latency denotes the temporal delay experienced while transferring task data from the user device to the MEC server. This latency significantly influences task offloading performance by affecting the time required to upload task data for subsequent processing and computation at the edge server. It can be expressed as:

$$T_i^{tran}={\displaystyle \frac{{\omega }_i^{mec}+u_i}{r_i}},$$

(6)

where $u_i$ indicates the size of the redundant data added when offloading the task. Based on the aforementioned equation, the offloading latency can be expressed as:

$$T_i^{offload}=T_i^{comp}+T_i^{tran}.$$

(7)

As the task undergoes partial offloading to the MEC server, the synchronization of local computation and offloading tasks becomes crucial. Consequently, the latency associated with offloading to the MEC server is determined by taking the maximum value between the local computation latency and the offloading latency. This yields the total latency for offloading to the edge server, which can be expressed as [13]:

$$\begin{array}{c}{T}_i=\underset{{\omega }_i^{mec}}{max}\{T_i^{comp}+T_i^{tran},T_i^{loc}\}.\hfill \end{array}$$

(8)

Similarly, we can deduce the energy consumption entailed in offloading tasks to the edge server. Let $E_i^{offload}$ denote the energy consumption attributed to the MEC server computation. In the scenario of offload computing, the principal driver of energy consumption is the transmission energy cost denoted as $E_i^{tran}$, which arises from the offloading process of the task to the MEC server,

$$E_i^{offload}=E_i^{tran}={\tau }_i·T_i^{tran}={\tau }_i{\displaystyle \frac{{\omega }_i+u_i}{r_i}}.$$

(9)

In summary, the energy consumption associated with processing the task generated by user i can be expressed as:

$$E_i=E_i^{loc}+E_i^{offload}.$$

(10)

To provide users with low-latency, low-bandwidth task offloading services, we combine weights ${\omega }_t$ and ${\omega }_e$ to define the objective function as

$$C={\phi }_t{T}_i+{\phi }_e{E}_i.$$

(11)

4. Privacy Protection Model and Problem Formulation

4.1. Location Privacy Leakage

In the process of MEC task offloading, the wireless channel power gain is highly correlated with the distance between users and MEC servers [18]. Users determine their offloading strategy, specifically the volume of offloaded tasks, based on the wireless channel conditions between themselves and the MEC servers. Consequently, attackers can infer channel state information merely by analyzing the offloading patterns.

Differential privacy introduces random noise to the data prior to its release, thereby obfuscating the users’ true locations. This approach effectively prevents attackers from tracking the data source [29]. However, traditional differential privacy methods based on the Laplace mechanism face the challenge of directly protecting users’ geographic location privacy in task offloading scenarios. Therefore, we construct a distance probability density function (PDF) that perturbs the distance l to $l^{*}\in [l_1,l_2]$, as shown in Figure 1. The PDF can be defined as:

$$PDF\left(l^{*}|l\right)={\displaystyle \frac{ϵ}{2\Delta l}}{e}^{-{\displaystyle \frac{ϵ(l^{*}-l)}{\Delta l}}}+{\displaystyle \frac{e^{\frac{ϵ(l_1-l)}{\Delta l}}+e^{-\frac{ϵ(l_2-l)}{\Delta l}}}{2\Delta l}}.$$

(12)

Given that the perturbation of the true distance through differential privacy is inherently stochastic, there may be cases where the true distance l and the perturbed distance $l^{*}$ are equal. A higher probability of these distances being equal indicates an increased risk of user privacy leakage, which is an unavoidable consequence. To quantify the alignment between the actual distance under privacy protection and the true distance without protection, we use the Kullback–Leibler Divergence (KLD) [30]. The specific calculation method is detailed as follows:

$$\mathrm{KLD}\left(P\parallel Q\right)={\int }_{l_1}^{l_2}Q\left(l\right)ln{\displaystyle \frac{Q\left(l\right)}{PDF\left(l^{*}\right|l)}}d{l}^{*},$$

(13)

where $Q\left(l\right)$ represents the probability distribution of user devices performing task offloading calculations based on the true distance without privacy protection services. In this study, we assume that the distribution of user locations is uniformly distributed within the coverage range of the MEC server, denoted as $Q\left(l\right)=1/l_{max}$. $PDF\left(l^{*}|l\right)$ denotes the probability distribution of user tasks being offloaded based on the perturbed distance after interference.

According to the definition of KLD, a smaller value of the fitted KLD indicates a higher degree of fit between the two probability distributions. This implies a greater likelihood of user privacy leakage. Conversely, a larger KLD value suggests a lower degree of fit and a reduced probability of user privacy leakage. Therefore, we define privacy leakage (PL) as the negative of KLD:

$$PL=-{\int }_{l_1}^{l_2}Q\left(l\right)ln{\displaystyle \frac{Q\left(l\right)}{PDF\left(l^{*}|l\right)}}d{l}^{*}.$$

(14)

4.2. Problem Formulation

The objective of user offloading computation tasks is defined as follows: under the precondition of ensuring a specified level of location privacy leakage, to derive optimal task offloading decisions (i.e., optimizing the volume of offloaded tasks). This aims to minimize the cost of user task offloading, thereby reducing the overall cost that incorporates location privacy protection.

$$\begin{array}{c}\underset{{\omega }_i^{mec}}{min}C={\phi }_t{T}_i+{\phi }_e{E}_i,\hfill \\ s.t.{\phi }_t+{\phi }_e=1,\hfill \\ 0<{\omega }_i^{mec}<{\omega }_i.\hfill \end{array}$$

(15)

The determination of interference ranges $l_1$ and $l_2$ is crucial for applying differential privacy perturbation to the confusion distance. Generally, when the true distance is relatively small, the fluctuation of the perturbed distance is correspondingly minor. This ensures that users allocate more computational tasks to the MEC server, thereby saving offloading costs. Therefore, to ensure that the task offloading decisions based on the perturbed distance $l^{*}$ are comparable to those made at distance l in terms of offloading cost (i.e., minimizing the degradation cost of service quality due to the perturbed distance), the value of $l^{*}$ should be within the smallest possible range around l. The closer $l_1$ and $l_2$ are to l, the better the practicality of the optimized offloading service after privacy protection.

When the range $▵l$ between $l_1$ and $l_2$ increases, the level of user privacy leakage decreases, indicating better effectiveness of the privacy protection mechanism; conversely, it decreases. However, when $▵l$ becomes too large, the perturbation caused by distance confusion in task offloading costs can lead to deviations from the optimal cost at l, ultimately resulting in decreased utility of task offloading decisions due to the confusion between distances. Therefore, to ensure the performance loss in the privacy-protected offloading process is minimized, the optimization objective adjusting the lower bound $l_1$ and upper bound $l_2$ of the confusion range is formulated as follows:

$$\begin{array}{cc}\hfill & \underset{l_1,l_2}{min}\mathit{PL},\hfill \\ \hfill & \mathit{s}.\mathit{t}.{\displaystyle \frac{\widehat{C}}{E\left(\widehat{C^{*}}\right)}}>\theta ,\hfill \end{array}$$

(16)

where $\theta$ represents the task loss threshold and $\widehat{C}$ denotes the offloading cost corresponding to the optimal offloading strategy at the user’s original position. $E\left(\widehat{C^{*}}\right)$ denotes the expectation of the offloading cost corresponding to the optimal offloading strategy with the obfuscated distance. It can be obtained via the following equation:

$$E\left(\widehat{C^{*}}\right)={\int }_{l_1}^{l_2}\widehat{C^{*}}\left(l^{*}\right)PDF\left(l^{*}|l\right)d{l}^{*}.$$

(17)

In this study, the offloading strategy is publicly determined by the MEC server, meaning the server’s resource allocation strategy is known.

4.3. Theoretical Analysis

In this section, we validate that the probability density function (PDF) used in this paper can obfuscate distances through differential privacy, thereby protecting the location privacy of tasks.

Theorem 1. 

When the obfuscation range is $[l_1,l_2]$, the total probability within this range for our PDF is 1.

Proof. 

The total probability within $[l_1,l_2]$ for our PDF is:

$$\begin{array}{c}{\int }_{l_1}^{l_2}{\displaystyle \frac{\epsilon }{2\Delta l}}{e}^{-{\displaystyle \frac{\epsilon \left|l^{*}-l\right|}{\Delta l}}}+{\displaystyle \frac{e^{\frac{\epsilon \left(l_1-l\right)}{\Delta l}}+e^{-\frac{\epsilon \left(l_2-l\right)}{\Delta l}}}{2\left(l_2-l_1\right)}}d{l}^{*}\hfill \\ ={\int }_{l_1}^l{\displaystyle \frac{\epsilon }{2\Delta l}}{e}^{{\displaystyle \frac{\epsilon \left(l^{*}-l\right)}{\Delta l}}}+{\displaystyle \frac{e^{\epsilon \frac{\left(l_1-l\right)}{\Delta l}}+e^{-\frac{\epsilon \left(l_2-l\right)}{\Delta l}}}{2\left(l_2-l_1\right)}}d{l}^{*}+{\int }_l^{l_2}{\displaystyle \frac{\epsilon }{2\Delta l}}{e}^{-{\displaystyle \frac{\epsilon \left(l^{*}-l\right)}{\Delta l}}}+{\displaystyle \frac{e^{\frac{\epsilon \left(l_1-l\right)}{\Delta l}}+e^{-\frac{\epsilon \left(l_2-l\right)}{\Delta l}}}{2\left(l_2-l_1\right)}}d{l}^{*}\hfill \\ =\left({\displaystyle \frac{1}{2}}{e}^{{\displaystyle \frac{\epsilon \left(l^{*}-l\right)}{\Delta l}}}+{\displaystyle \frac{e^{\frac{\epsilon \left(l_1-l\right)}{\Delta l}}+e^{-\frac{\epsilon \left(l_2-l\right)}{\Delta l}}}{2\left(l_2-l_1\right)}}·l^{*}\right)\left|{ }_{l_1}^l\right.+\left(-{\displaystyle \frac{1}{2}}{e}^{-{\displaystyle \frac{\epsilon \left(l^{*}-l\right)}{\Delta l}}}+{\displaystyle \frac{e^{\frac{\epsilon \left(l_1-l\right)}{\Delta l}}+e^{-\frac{\epsilon \left(l_2-l\right)}{\Delta l}}}{2\left(l_2-l_1\right)}}·l^{*}\right)\left|{ }_l^{l_2}\right.\hfill \\ ={\displaystyle \frac{1}{2}}+{\displaystyle \frac{e^{\frac{\epsilon \left(l_1-l\right)}{\Delta l}}+e^{-\frac{\epsilon \left(l_2-l\right)}{\Delta l}}}{2\left(l_2-l_1\right)}}·l-{\displaystyle \frac{1}{2}}{e}^{{\displaystyle \frac{\epsilon \left(l_1-l\right)}{\Delta l}}}-{\displaystyle \frac{e^{\frac{\epsilon \left(l_1-l\right)}{\Delta l}}+e^{-\frac{\epsilon \left(l_2-l\right)}{\Delta l}}}{2\left(l_2-l_1\right)}}·l_1-{\displaystyle \frac{1}{2}}{e}^{{\displaystyle \frac{\epsilon \left(l_2-l\right)}{\Delta l}}}\hfill \\ +{\displaystyle \frac{e^{\frac{\epsilon \left(l_1-l\right)}{\Delta l}}+e^{-\frac{\epsilon \left(l_2-l\right)}{\Delta l}}}{2\left(l_2-l_1\right)}}·l_2+{\displaystyle \frac{1}{2}}-{\displaystyle \frac{e^{\frac{\epsilon \left(l_1-l\right)}{\Delta l}}+e^{-\frac{\epsilon \left(l_2-l\right)}{\Delta l}}}{2\left(l_2-l_1\right)}}·l\hfill \\ =1\hfill \end{array}$$

Clearly, the total probability of the PDF used in this paper within the obfuscation range $[l_1,l_2]$ is 1. This confirms that the user’s obfuscation distance does not exceed the obfuscation range, demonstrating that the proposed obfuscation range $[l_1,l_2]$ is reasonable. Furthermore, we will now verify the security of the location privacy protection mechanism proposed in this paper.   □

Theorem 2. 

The location privacy protection mechanism proposed in this study adheres to ϵ-differential privacy.

Proof. 

$$\begin{array}{c}{\displaystyle \frac{Pr\left(l^{*}\right|l)}{Pr\left(l^{*}\right|l^{\prime })}}={\displaystyle \frac{\frac{\epsilon }{2\Delta l}{e}^{-\frac{\epsilon \left|l^{*}-l\right|}{\Delta l}}+\frac{e^{-\frac{\epsilon \left|l_2-l\right|}{\Delta l}}+e^{\frac{\epsilon \left|l_1-l\right|}{\Delta l}}}{2\Delta l}}{\frac{\epsilon }{2\Delta l}{e}^{-\frac{\epsilon \left|l^{*}-l^{\prime }\right|}{\Delta l}}+\frac{e^{-\frac{\epsilon \left|l_2-l^{\prime }\right|}{\Delta l}}+e^{\frac{\epsilon \left|l_1-l^{\prime }\right|}{\Delta l}}}{2\Delta l}}}\le max\left({\displaystyle \frac{e^{-\frac{\epsilon \left|l^{*}-l\right|}{\Delta l}}}{e^{-\frac{\epsilon \left|l^{*}-l^{\prime }\right|}{\Delta l}}}},{\displaystyle \frac{\frac{e^{-\frac{\epsilon \left|l_2-l^{\prime }\right|}{\Delta l}}+e^{\frac{\epsilon \left|l_1-l^{\prime }\right|}{\Delta l}}}{\epsilon }}{\frac{e^{-\frac{\epsilon \left|l_2-l\right|}{\Delta l}}+e^{\frac{\epsilon \left|l_1-l\right|}{\Delta l}}}{\epsilon }}}\right)\\ \le max\left(e^{-{\displaystyle \frac{\epsilon \left|l^{\prime }-l\right|}{\Delta l}}},max\left({\displaystyle \frac{e^{-\frac{\epsilon \left|l_2-l\right|}{\Delta l}}}{e^{-\frac{\epsilon \left|l_2-l^{\prime }\right|}{\Delta l}}}},{\displaystyle \frac{e^{\frac{\epsilon \left|l_1-l\right|}{\Delta l}}}{e^{\frac{\epsilon \left|l_1-l^{\prime }\right|}{\Delta l}}}}\right)\right)\le max\left(e^{\epsilon },max\left(e^{\epsilon },e^{\epsilon }\right)\right)=e^{\epsilon }\end{array}$$

In summary, the proposed obfuscation range in this paper is reasonable, and the location privacy protection mechanism presented herein demonstrates a high level of security.   □

5. DQN-Based Privacy Protection Algorithm for Differential Location Obfuscation

To protect the privacy of user tasks, we quantified this aspect in Section 4 and further proposed the optimization objective of our paper—to minimize the processing cost of user tasks (in terms of latency and energy consumption) while ensuring privacy protection. In this section, we will introduce the specific solution method, with the corresponding framework illustrated in Figure 2.

In DQN, there are two types of networks. The evaluation network is the primary operational network in DQN; it outputs the Q-values for each possible action given the current state. This network is used to update the Q-value function and learns the optimal policy by repeatedly adjusting weights based on training data. The target network’s role is to generate stable target Q-values, which mitigates the fluctuations of the evaluation network during training. In the training process of DQN, the evaluation network continuously learns and updates, functioning as a dynamic network, while the target network, as a relatively static network, provides a more stable target value. This dual-network structure effectively enables DQN to converge more robustly to the optimal policy. In the MEC offloading optimization problem addressed in this paper, the definitions for state, action, and reward are as follows:

• State: The system state is designed as follows: the task status of user requests, that is, the task request parameters of each user within the MEC coverage area, can be represented by a one-dimensional vector: $s_t=[{\omega }_1,{\omega }_2,...,{\omega }_n]$, where ${\omega }_i$ denotes the total task size generated by user i.
• Action: The proposed solution addresses the task offloading size. Thus, we have designed the task set as:

  $$A=\{a_1,a_2,...,a_n\},$$

  (18)

  where $a_i$ indicates the offloading decision of user i, which is the offloading task quantity.
• Reward: In this study, the optimization objective involves minimizing the system’s cost, whereas the learning objective in reinforcement learning aims to maximize the benefits obtained. To effectively utilize reinforcement learning in problem solving, it is essential to establish a connection between the reward function and the optimization objective function. Consequently, the reward value is defined as the negation of the optimization objective function. By doing so, the reinforcement learning agent seeks to maximize the reward, thereby aligning its actions with the overarching objective of cost minimization.

The primary objective of the Deep Q-Network (DQN) is to estimate the expected reward associated with an action, which is referred to as the Q value. DQN surpasses conventional reinforcement learning algorithms that rely on tabular representations by leveraging a deep neural network architecture to process the state information of the edge nodes. This enables the computation of Q-values for all feasible actions, eliminating the need for maintaining explicit Q-value tables. Consequently, the issue of managing an infinite state space is effectively mitigated. To facilitate exploration during the learning process, the $\epsilon$-Greedy strategy is adopted, whereby a random action is chosen with a probability of p, as defined as

$$e=\left\{\begin{array}{c}rand\left(a\right)\\ argmaxQ(s,a)\end{array}\right.\begin{array}{c}p\in [0,\epsilon )\\ p\in [\epsilon ,1]\end{array}$$

Exploration through the $\epsilon$-Greedy strategy serves to prevent the algorithm from converging to suboptimal solutions. In this study, a decreasing $\epsilon$-Greedy strategy is employed, whereby the exploration parameter decreases as the number of training iterations progresses. This approach facilitates thorough exploration of action possibilities during the initial stages of training, while gradually reducing the likelihood of selecting random actions in later stages, thus enhancing the stability of the algorithm.

As shown in Figure 2, the DQN framework incorporates two neural networks for training purposes: an evaluation network, which is trained, and a target network responsible for generating Q values. The training of the model parameters is achieved through the utilization of a loss function. Specifically, the loss function represents the squared difference between the output of the primary network and the estimation produced by the target network. The mathematical expression for the loss function is as follows:

$$LOSS={(Q_r(s,a)-Q(s,a))}^2,$$

where the estimated value of the target network $Q_r(s,a)$ is given as

$$Q_r(s,a)=\mathrm{reward}+\gamma Q^{\prime }\left(s^{\prime },arg\underset{a^{\prime }}{max}{Q}^{\prime }(s^{\prime },a^{\prime })\right),$$

where $\gamma$ is the decay coefficient applied to the reward in the next step, $Q^{\prime }(s,a)$ is the estimate of the reward in the target network for performing action a in state s, and $arg{max}_{a^{\prime }}{Q}^{\prime }(s^{\prime },a^{\prime })$ is the action $a^{\prime }$ corresponding to the highest Q-value in the next state $s^{\prime }$.

In this paper, the above DQN methodology is applied to obtain the optimal offloading solution with a given distance, with the training process of DQN shown in Algorithm 1. Through traversing the obfuscation interval, the privacy leakage minimized offloading strategy with an optimal obfuscation interval is achieved, and the detailed process is provided in Algorithm 2.

Algorithm 1 Training process of DQN.

1: Input: System state space at time t 2: Output: Offloading decisions and resource allocation decisions 3: Initialize the parameters. 4: for each episode = 1, 2, …, N do 5:     Observe the system environment to obtain the current state s 6:     for each time slot t do 7:         if rand(0,1) $<ϵ$ then 8:            Calculate the optimal action a 9:         else 10:            Randomly select action a 11:         end if 12:     end for 13:     Execute action a and calculate feedback rewards based on the reward function 14:     Obtain the next state $s^{\prime }$ and r 15:     Output the decision list 16: end for 17: Set the iteration rounds M and parameter replacement interval $\delta$ 18: for each iteration round i = 1, 2, …, M do 19:     for each $(s,a,r,s^{\prime })$ randomly do 20:         Calculate the $Q(s,a)$, $Q_r(s,a)$, $LOSS$ 21:         if episode $\%\delta ==0$ then 22:            Replace($Q_r(s,a)$, $Q(s,a)$) 23:         end if 24:     end for 25: end for

Algorithm 2 DQN-based privacy protection algorithm with differential privacy (DQN-DP).

1: Input: $l_{max},l,{\omega }_i,H\left(t\right),{\varphi }_1,{\varphi }_2$ 2: Output: $[l_1,l_2],{\omega }_i^{mec}$ 3: Calculate $\widehat{C}$ based on l and Algorithm 1 4: $l_1=0,l_2=l_{max}$ 5: while $l_1\ne l_2$do 6:     $l_1=l_1+{\varphi }_1,l_2=l_2-{\varphi }_2$ 7:     Calculate $PL$ based on Equation (14) 8:     for $l^{*}$ do 9:         Calculate $E\left(\widehat{C^{*}}\right)$ using Algorithm 1 10:         Based on offloading task cost reduction $\widehat{C}/E\left(\widehat{C^{*}}\right)$, calculate $[l_1,l_2]$ 11:         if $\widehat{C}/E\left(\widehat{C^{*}}\right)>\theta$ then 12:            Record $[l_1,l_2]$ in $Interval$ 13:         end if 14:     end for 15: end while 16: Choose optimal $[l_1,l_2]$ from $Interval$ to minimize $PL$ 17: Randomly select $l^{*}$ according to the probability density function given by Equation (12), and determine the offloading task size based on Algorithm 1

6. Numerical Analysis

To validate the reliability and efficiency of the DQN-DP algorithm, we created a simulation environment. In our simulated experiment, an edge server with a coverage area of approximately 300 m was established. Within this coverage area, we set up 10 edge nodes as users. Bandwidth was uniformly allocated based on the number of users. Furthermore, we initially selected a power gain that followed an exponential distribution with a parameter of 1, with the $\theta$ accounting for 80% of the total. The network parameter settings for the experimental environment in this chapter are detailed in

Table 2. Parameter values.

Parameters	Value
Processing frequency at the user $f_i^{loc}$	1 (GHz)
Processing frequency at MEC server$f_i^{mec}$	10 (GHz)
Bandwidth of the wireless channel B	$1\times {10}^6$ (Hz)
Number of edge devices K	10
Computing density $\beta$	1000 cycles/bit
Number of hidden layer nodes	50
Learning rate	0.001
Discount rate	0.9
Experience pool size	300
Batch size	64
Path loss constant $g_0$	20 dB
Path loss exponent $\vartheta$	4 dB/m

Source: This table provides the parameter values used in the experiment.

.

The simulation experiments in this section are divided into two main parts. First, we validate the feasibility of the DQN-DP algorithm by measuring the extent of privacy leakage. Then, we assess the superiority of the DQN-DP algorithm in terms of resource consumption compared to other algorithms. To facilitate reader understanding, we will provide a brief introduction to the comparison algorithms as follows:

• Basic: This algorithm focuses on basic task offloading without considering user privacy, and the optimization objective is to minimize the terminal energy consumption for each time slot t. In a sense, it is somewhat similar to the Greedy algorithm.
• Max Range: This algorithm does not select the obfuscation range $[l_1,l_2]$ to strike a balance between cost loss and privacy protection. Task offloading is then performed using the obfuscated location without considering the interference range.
• CDMP [18]: This algorithm aims to minimize device energy consumption as its optimization objective. A privacy protection level P is predefined in the algorithm, where a higher P indicates stronger privacy protection and vice versa. In our comparative analysis, we set the value of P to be $1/3{\omega }_i$.

6.1. Reliability Analysis

We first validated the rationale of the DQN-DP algorithm through experiments. As shown in Figure 3, as the distance between the user and the MEC increases, the offloading cost gradually rises, leading to an increase in the local execution rate of tasks. In other words, the user’s local execution rate and offloading cost both increase with the distance from the MEC. This occurs because the channel conditions between the user and the MEC server are inversely proportional to the distance between them. Consequently, users tend to process more tasks locally, resulting in higher computation costs. This observation reveals that the curves of local execution rate and cost are convex functions that gradually converge, demonstrating that the DQN-DP algorithm can find the optimal solution across various scenarios with different channel parameters and distances.

To further validate that the DQN-DP algorithm effectively identifies the optimal local execution ratio, we evaluate the obtained solution as illustrated in Figure 4. Clearly, at a distance of 150, the optimal local execution ratio derived from the DQN-DP algorithm results in a cost that is lower than the cost associated with other local execution ratios. This indicates that the local execution ratio determined by the DQN-DP algorithm is indeed the one that minimizes cost. Figure 5 illustrates the relationship between the distance from users to the MEC servers and the obfuscation interval. As the distance between the MEC server and the user device increases, both the lower bound $l_1$ and the upper bound $l_2$ of the obfuscation interval expand around the true distance l. This expansion is centered on the true distance l. The selection of the obfuscation interval is determined by a constraint involving the trade-off between the expected optimal offloading strategy cost after obfuscation and the original task offloading optimization cost. As such, under the optimization constraints of this study, user devices choose an appropriate obfuscation interval based on the distance. The fluctuation in the difference between the bounds of the obfuscation range $[l_1,l_2]$ remains relatively stable and does not exhibit significant variability as the distance changes. Therefore, we find that the DQN-DP algorithm is stable. In summary, DQN-DP is capable of finding the optimal offloading ratio across various parameter settings and can consistently determine the optimal confusion range in environments with varying distances. Its reliability has been demonstrated.

6.2. Efficiency Analysis

Furthermore, we evaluate the cost comparison between the DQN-DP algorithm and other algorithms under various conditions. Figure 6 presents a performance comparison of the Basic, Max Range, CMDP, and DQN-DP algorithms in terms of average task offloading cost as the distance between user devices and MEC servers gradually increases. As observed from the graph, the average cost consumption of these three mechanisms increases with distance. This is attributed to the deterioration of the wireless channel conditions as distance increases, leading to an increased frequency of local task processing by users. Moreover, the DQN-DP algorithm proposed in this study exhibits costs relatively close to the CMDP algorithm after the distance reaches 150 m. To further validate the superiority of DQN-DP, we compare the offloading costs of various solutions under different path loss constants, as shown in Figure 7. It is evident that, in all environments, DQN-DP consistently outperforms both the Max Range and CMDP algorithms, falling just short of the Basic algorithm, which does not consider privacy protection.

The Max Range algorithm provides a larger obfuscation range to minimize privacy leakage but does not account for the relationship between privacy leakage and task offloading cost. By obfuscating based on the maximum resource boundary of the edge server coverage, it fails to consider the potential increase in offloading cost resulting from location obfuscation. In contrast, the DQN-DP algorithm balances privacy protection with offloading cost. It does so by considering the ratio of offloading costs under two strategies: the privacy protection strategy and the efficiency-first strategy. This ratio is used as an optimization constraint. As a result, the DQN-DP algorithm manages to keep its cost between those of the Basic and Max Range algorithms.

Since the DQN-DP algorithm is a privacy protection algorithm, we need to compare the privacy leakage (PL) of various algorithms. Figure 8 shows the comparison of the privacy leakage (PL) values for different algorithms as the distance between user devices and the MEC server increases. The Basic algorithm, which offloads tasks based on true location without considering privacy protection, aligns perfectly with the KLD fitting function, resulting in a privacy leakage level of 0.

It can be observed that the DQN-DP algorithm maintains a stable privacy leakage level between that of the Basic and Max Range algorithms. This stability arises from the optimization constraint imposed by the DQN-DP algorithm during the location privacy protection process. As the distance between the user and the MEC server increases, the constraint ensures that the expected cost of the optimal offloading for each random point within the obfuscation range does not exceed 80% of the cost of the optimal offloading at the original location. Consequently, the privacy leakage level remains relatively stable despite changes in distance. On the other hand, the Max Range algorithm does not strike a balance between task computation cost and privacy protection during obfuscation, resulting in the lowest privacy leakage level.

Figure 9 illustrates the comparison of task offloading costs among algorithms under different task cost loss ratios. The Basic algorithm, which does not consider privacy protection, remains unaffected by the cost loss ratio parameter. Due to the Max Range algorithm’s lack of balance between task computation cost and privacy protection during obfuscation, it incurs higher offloading costs across varying cost loss ratio conditions. By offloading tasks within the maximum MEC coverage range, the Max Range algorithm consistently exhibits elevated offloading costs under different cost loss ratio conditions. As the value of the task cost loss ratio increases, the DQN-DP algorithm places greater emphasis on the cost loss incurred by the obfuscation distance during obfuscation range selection. Consequently, the task cost shows a decreasing trend with increasing cost loss ratio. At a cost loss ratio of 75%, the task cost of the DQN-DP algorithm approaches that of the CMDP algorithm. Moreover, the task offloading utility of the DQN-DP algorithm surpasses that of the CMDP algorithm beyond the 75% threshold. In summary, DQN-DP is a privacy protection algorithm with a high cost-performance ratio. Compared to the Basic algorithm, which has lower cost but lacks privacy protection, DQN-DP effectively safeguards against privacy leakage. In comparison to privacy-protecting algorithms like Max Range and CMDP, DQN-DP offers lower overhead and stable protection performance.

7. Conclusions

To address the issue of location privacy leakage during the MEC network offloading process, this study investigates task scheduling strategies with privacy protection features. Initially, we model the task offloading scenario with a single MEC server, employing both real and obfuscated distances to determine the confusion range. Subsequently, based on the extent of privacy leakage concerning user location, we propose the DQN-DP algorithm based in DQN to solve the optimal offloading problem under given distance and offloading cost constraints. Theoretical analysis shows that the proposed probability density function has a probability of 1 within the confusion range, and the DQN-DP algorithm adheres to $ϵ$-differential privacy, thus effectively ensuring privacy protection. Through simulation, we observe that the DQN-DP algorithm provides a better trade-off between overhead and privacy leakage compared to the baseline approach. In summary, the DQN-DP algorithm effectively reduces privacy leakage and offloading costs in MEC networks, representing a balanced privacy protection solution. Given the complexity of edge topologies, our current study uses distance as the sole confusion metric, which is not comprehensive for practical applications. Therefore, in future work, we will explore privacy protection schemes using node coordinates as the confusion metric.